Method and device for correcting nonlinear distortion of image and storage medium

Abstract

The invention discloses an image nonlinear distortion correction method and device and a storage medium, which are used for eliminating nonlinear distortion and improving the detection efficiency and quality. The method comprises the steps that a checkerboard image is input into a target image, the target image is segmented into a plurality of sub-regions according to the checkerboard image, and the target image is an image generating nonlinear distortion; extracting sub-region corner coordinates of the sub-regions; sorting the angular point coordinates of the sub-regions; according to the sub-region corner coordinates, the black-and-white lattice corner coordinates of the checkerboard image and the sorting result, a homography transformation matrix is obtained through calculation, and the black-and-white lattice corner coordinates are standard corner coordinates of a plurality of black-and-white lattices in the checkerboard image; and converting the target image into a corrected image through the homography transformation matrix.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device and storage medium for correcting nonlinear image distortion. Background Art

[0002] With the rapid development of virtual reality (VR) technology, creating a highly realistic, comfortable, and immersive virtual experience for users has become a core goal of the industry. However, nonlinear distortion is a common and critical issue in VR scenarios that needs to be addressed urgently.

[0003] From an optical perspective, the structural characteristics of short-focus lenses dictate that their curvature is significantly greater than that of normal lenses. When light passes through a lens with such a large curvature, its path of propagation changes dramatically. This is especially true at the edges of the lens, where the greater curvature causes more intense refraction. This causes the image formed by the lens to be significantly distorted at the edges, deviating from the normal image and resulting in nonlinear distortion.

[0004] Furthermore, in the optical system of a VR headset, the display screen is placed close to the lens. Pixels at different locations on the screen, after being refracted by the lens, exhibit a complex nonlinear mapping relationship on the final imaging plane, resulting in distortion and deformation. This nonlinear mapping relationship exacerbates the degree of nonlinear image distortion, resulting in noticeable distortion and deformation in the images displayed through the VR headset.

[0005] This nonlinear distortion not only affects the user's viewing experience, but also during the defect detection process of VR displays, because the AOI optical system uses conventional lenses, obvious pincushion distortion will appear when shooting the VR screen, making defect detection impossible. Summary of the Invention

[0006] The present application discloses a method, device and storage medium for correcting nonlinear image distortion, which are used to eliminate nonlinear distortion and improve detection efficiency and quality.

[0007] The first aspect of the present application discloses a method for correcting nonlinear image distortion, comprising: A checkerboard image is input into a target image, and the target image is divided into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that produces nonlinear distortion; sub-region corner point coordinates of the sub-regions are extracted; the sub-region corner point coordinates are sorted; a homography transformation matrix is ​​calculated according to the sub-region corner point coordinates, the black and white grid corner point coordinates of the checkerboard image, and the sorting result, wherein the black and white grid corner point coordinates are the standard corner point coordinates of a plurality of black and white grids in the checkerboard image; and the target image is converted into a corrected image using the homography transformation matrix.

[0008] Optionally, before sorting the sub-region corner point coordinates, the correction method further includes: Remove outliers in the coordinates of the sub-region corner points.

[0009] Optionally, removing abnormal points in the coordinates of the sub-region corner points includes: Set the row and column spacing thresholds between sub-regions; Sort the row coordinates of the corner point coordinates, and obtain the first row coordinate of the first corner point coordinate; Calculating first common row upper and lower limits according to the first row coordinates and the row spacing threshold, and obtaining a row coordinate array according to the first common row upper and lower limits; According to the row coordinate array, extract the corresponding column coordinate array; Calculating the values ​​in the column coordinate array to obtain a column coordinate difference array; If there is a column coordinate difference that is less than the column distance threshold, extract and summarize it into an abnormal point array; The elements in the outlier array are analyzed according to preset conditions, and corresponding outliers are removed according to the analysis results, wherein the elements in the outlier array include column coordinate differences that are less than the column distance threshold.

[0010] Optionally, analyzing the elements in the outlier array according to preset conditions and removing corresponding outliers according to the analysis results includes: If the first element of the column coordinate difference array exists in the element of the outlier array, then the sub-region corner point coordinates associated with the first element in the row coordinate array are removed; If the element in the outlier array contains the last element of the column coordinate difference array, then the sub-region corner coordinates associated with the next element after the corresponding element in the row coordinate array are removed; If the element in the outlier array is neither the first element nor the last element in the column coordinate difference array, obtaining the first difference between the corresponding element and the previous element in the row coordinate array; If the first difference is less than the column distance threshold, the sub-region corner coordinates associated with the corresponding element in the row coordinate array are removed; otherwise, the sub-region corner coordinates associated with the next element after the corresponding element in the row coordinate array are removed.

[0011] Optionally, the sorting of the sub-region corner point coordinates includes: Step 1: sorting the row coordinates in the row coordinate array and the column coordinates in the column coordinate array, and extracting the minimum row coordinate and the minimum column coordinate; Step 2: Obtaining the upper and lower limits of the same row and the same column corresponding to the minimum row coordinate and the minimum column coordinate according to the row spacing threshold and the column spacing threshold; Step 3: Record and store the row coordinates and column coordinates that satisfy the common row upper and lower limits and the common column upper and lower limits into a common row index set and a common column index set respectively; Step 4: removing the row coordinates and the column coordinates in the row coordinate array and the column coordinate array that have been stored in the co-row index set and the co-column index set; Step 5: Repeat steps 1 to 4 until all the row coordinates and column coordinates in the row coordinate array and the column coordinate array are traversed; Step 6: Traverse all common row index sets and common column index sets, and extract the index values ​​that appear in both common row index sets and common column index sets; Step 7: Based on the co-row index set and the co-column index set where the index value is located, the sub-region corner point coordinates corresponding to the index value are placed into a new coordinate set, wherein the length of the new coordinate set is the product of the number of rows and columns of black and white squares in the checkerboard image; Step 8: Repeat steps 6 and 7 until all sub-region corner point coordinates corresponding to the index values ​​are put into the new coordinate set.

[0012] Optionally, before step eight, the correction method further includes: Determine whether the number of the co-row index set or the co-column index set is less than the number of rows and columns of black and white squares in the checkerboard image; If yes, the coordinates of the missing corner points are calculated based on the co-row index set and the co-column index set; The missing corner point coordinates are put into the new coordinate set.

[0013] Optionally, calculating the missing corner coordinates according to the co-row index set and the co-column index set includes: Traverse the new coordinate set and record the row and column numbers of the missing corner point coordinates; According to the row and column numbers, respectively extracting the co-row index set and the co-column index set where the coordinates of the missing corner point are located; Calculating a co-row second-order fitting coefficient and a co-column second-order fitting coefficient according to the co-row index set and the co-column index set; The coordinates of the missing corner points are calculated based on the co-row second-order fitting coefficients and the co-column second-order fitting coefficients.

[0014] A second aspect of the present application provides a device for correcting nonlinear image distortion, comprising: a segmentation unit, configured to input a checkerboard image into a target image and segment the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that generates nonlinear distortion; An extraction unit, configured to extract the coordinates of the sub-region corner points of the sub-region; A sorting unit, configured to sort the coordinates of the sub-region corner points; a calculation unit, configured to calculate a homography transformation matrix based on the coordinates of the corner points of the sub-regions and the coordinates of the corner points of the black and white squares of the checkerboard image, wherein the coordinates of the corner points of the black and white squares are standard corner point coordinates of a plurality of black and white squares in the checkerboard image; A conversion unit is configured to convert the target image into a rectified image using the homography transformation matrix.

[0015] A third aspect of the present application provides a device for correcting nonlinear image distortion, comprising: processor, memory, input and output units, and buses; The processor is connected to the memory, input and output units, and the bus; The memory stores a program, and the processor calls the program to execute the first aspect and any optional correction method of the first aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional correction method of the first aspect.

[0017] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: For nonlinear display distortion, the regular grid of the checkerboard image can conform to the distortion pattern, dividing the target image into subregions that are tailored to the distortion trend. By breaking down complex global nonlinear distortion into relatively controllable local subregion distortions, the subsequent correction difficulty is reduced and the correction is more targeted. The coordinates of the subregion corner points contain spatial position information and clearly reflect the arrangement order of the subregions in the target image. Extracting the coordinates of the subregion corner points helps quantify the subregion distortion state and provides data support for the subsequent calculation of the correction matrix.

[0018] Sorting the coordinates of the subregion corner points establishes an accurate correspondence between them and the standard checkerboard corner points. Sorting also corrects the spatial dislocation of the corner points caused by nonlinear distortion, ensuring that the corner coordinates strictly adhere to the row and column logic of the checkerboard. This ensures that when the homography matrix is ​​subsequently calculated, the distorted corner points precisely correspond to the standard corner points, avoiding correction errors caused by coordinate mismatches.

[0019] The homography matrix is ​​calculated by using the coordinates of the sub-region corner points, the coordinates of the standard black and white grid corner points, and the sorting results. The homography matrix accurately describes the conversion rules between the distorted corner points and the standard corner points. Using the calculated homography matrix to transform the target image effectively eliminates distortion, converting the distorted image into a normal, clear image that most closely resembles the actual display screen state. This enables the defect detection system to accurately identify defects such as scratches, bright spots, and dark spots on the display, improving inspection efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A schematic diagram of an embodiment of a method for correcting nonlinear image distortion in this application; Figure 2 A schematic diagram of an embodiment of a method for removing outliers in this application; Figure 3 A schematic diagram of an embodiment of a method for removing outliers based on analysis results in this application; Figure 4 Schematic diagram of an embodiment of a method for sorting sub-region corner point coordinates in the present application; Figure 5 A schematic diagram of an embodiment of a method for supplementing missing corner point coordinates in this application; Figure 6 A schematic diagram of an embodiment of a method for calculating the coordinates of missing corner points in this application; Figure 7 A schematic diagram of the structure of the device for correcting nonlinear image distortion in this application; Figure 8 This is another structural schematic diagram of the device for correcting nonlinear image distortion in this application; Figure 9 A schematic diagram of nonlinear distortion of an image in this application; Figure 10 A schematic diagram of a checkerboard image in this application; Figure 11 A schematic diagram of identifying sub-region corner points in this application; Figure 12 This is a schematic diagram of the abnormal corner coordinate types that appear after severe distortion; Figure 13This is a schematic diagram of corner points not extracted in this application; Figure 14 A schematic diagram of the intersection of two parabolas and the edge of an image in this application; Figure 15 A schematic diagram of filling in missing corner coordinates in this application. DETAILED DESCRIPTION

[0022] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0023] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0024] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0025] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0026] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0028] In the optical system of a VR headset, the display is placed close to the lens. Pixels at different locations on the screen are refracted by the lens, resulting in a complex nonlinear mapping relationship on the final imaging plane. This nonlinear mapping relationship exacerbates the nonlinear distortion of the image, causing noticeable distortion and deformation in the image displayed through the VR headset.

[0029] Nonlinear distortion includes barrel distortion and pincushion distortion, such as Figure 9 As shown, Figure 9 A schematic diagram of nonlinear image distortion. This nonlinear distortion not only affects the user's viewing experience, but also, during defect inspection of VR displays, significant pincushion distortion occurs when capturing VR screens, making defect detection impossible because AOI optical systems use conventional lenses.

[0030] Based on this, the present application discloses a method for correcting nonlinear image distortion, which is used to eliminate nonlinear distortion and improve detection efficiency and quality.

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] The method of the present application can be applied to a server, device, terminal or other device with logic processing capability, and the present application does not limit this. For the convenience of description, the following description is based on the example of the execution subject being a system.

[0033] See also Figure 1 The present application provides an embodiment of a method for correcting nonlinear image distortion, comprising: 101. Inputting the checkerboard image into the target image, and dividing the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that produces nonlinear distortion; Checkerboard image Figure 10 As shown, the checkerboard image of the VR display is lit up by the PG (image generator), and the checkerboard image is input into the target image that produces nonlinear distortion. That is, the checkerboard is directly presented on the display to be detected as a reference template, forming a reference and detection relationship under the same carrier with the target image of the display itself.

[0034] The target image undergoes nonlinear distortion due to display characteristics, and the horizontal and vertical grid lines of the checkerboard image will deform synchronously with the target image's distortion. The target image is divided into several sub-regions, each corresponding to the display area covered by a single grid cell on the board, using the deformed grid lines as boundaries.

[0035] By segmenting, the originally distorted target image is decomposed into local sub-region distortions, and the degree of distortion in each sub-region is relatively uniform (for example, the central sub-region has a smaller distortion, and the edge sub-region has a consistent distortion trend).

[0036] 102. Extract the coordinates of the sub-region corner points of the sub-region; After chessboard segmentation, the boundary of each sub-region is composed of grid lines after the chessboard is deformed, and the corner point of the sub-region is the intersection point of two adjacent grid lines.

[0037] like Figure 11 As shown in the figure, after acquiring a target image containing a checkerboard pattern, an image processing algorithm is used to identify the corner points of the subregions. The algorithm locates the corner points based on the sudden change in the grayscale value of the pixels at the corner points. Then, based on the preset image coordinate system, the spatial position of the corner points is converted into specific coordinate values, ultimately obtaining the coordinates of the four corner points of each subregion.

[0038] 103. Sort the coordinates of the sub-region corner points; The ordering of sub-region corner coordinates is based on the original grid logic of the checkerboard. In an undistorted state, the checkerboard is arranged in regular rows and columns (e.g., columns from left to right, rows from top to bottom). Therefore, the ordering must be based on this inherent pattern and the spatial location of the corner coordinates. For example, sub-regions in the same row are sorted from smallest to largest by their horizontal coordinates (corresponding to a left-to-right order); sub-regions in the same column are sorted from smallest to largest by their vertical coordinates (corresponding to a top-to-bottom order). First, determine the base corner for sorting (e.g., the top-left corner of each sub-region). Then extract the coordinates of these base corners for all sub-regions. Sorting is then performed by coordinate comparison: groups are first grouped by vertical coordinate (those with similar vertical coordinates are placed in the same row), and then sorted within each group by horizontal coordinate. For edge sub-regions with coordinate shifts due to distortion, sorting is still based on the original grid's row and column logic, ignoring minor coordinate deviations caused by local distortion. By sorting, a one-to-one correspondence is established between the corner points of the sub-region and the standard corner points of the chessboard, avoiding the confusion of corner point positions caused by distortion and providing ordered matching data for the subsequent calculation of the homography transformation matrix.

[0039] 104. Calculate a homography transformation matrix based on the sub-region corner coordinates, the black and white grid corner coordinates of the checkerboard image, and the sorting result, where the black and white grid corner coordinates are standard corner coordinates of several black and white grids in the checkerboard image. The homography transformation matrix describes the mapping relationship between the target image (an image with nonlinear distortion) and the ideal undistorted image. The subregion corner coordinates represent the distorted corner locations of the target image, and their spatial correspondence has been determined through sorting. The checkerboard image's black and white square corner coordinates represent the standard corner locations of the ideal undistorted image and are known benchmark data. The sorting of the subregion corner coordinates ensures a one-to-one correspondence between the two sets of corner coordinates, avoiding misaligned matches. The subregion corner coordinates, the checkerboard image's black and white square corner coordinates, and the sorting result together constitute the matching pairs between the distorted and standard coordinates, providing the raw input for matrix calculations.

[0040] First, based on the sorting results, the subregion corner coordinates are paired with the corresponding checkerboard corner coordinates. For example, the subregion corner coordinates in row 3, column 2 after sorting will form a matching pair with the standard corner coordinates in row 3, column 2 of the checkerboard. The homography transformation matrix for each matching pair is then solved using the least squares method. The homography transformation matrix contains multiple parameters that quantitatively describe the transformation from the distorted corner coordinates to the standard corner coordinates, including translation, rotation, scaling, and nonlinear distortion correction.

[0041] Specific homography transformation matrix calculation process: First, define two sets of coordinates: the subregion corner coordinates {X', Y'} are the actual coordinates of the subregion corners in the distorted image; the checkerboard square corner coordinates {X, Y} are the standard coordinates under ideal conditions without distortion. The sorting result ensures that these two sets of coordinates are accurately matched according to the order of the corresponding checkerboard squares.

[0042] Based on these paired coordinates, a mathematical relationship is constructed using the formula:

[0043] The least squares method is used to solve the various parameters of the homography transformation matrix M, and it is determined that the homography transformation matrix M solved by the homography transformation matrix M can describe the transformation relationship from the distorted coordinates to the standard coordinates.

[0044] The calculated homography transformation matrix M is equivalent to dividing the corner points of each row or column of the distortion into several subregions. The position differences between them are different, but for each row or column of subregions, the change is basically linear, that is, the position difference between adjacent pixels remains consistent.

[0045] 105. Convert the target image into a rectified image through the homography transformation matrix.

[0046] For any pixel point in the target image (without the input checkerboard image), its coordinates {X', Y'} in the image coordinate system are combined with the homography transformation matrix M previously solved by the least squares method to map the pixel position in the distorted state to the ideal position without distortion.

[0047] Specifically, by traversing each pixel point of the target image, the homography transformation matrix M is used to calculate its corrected coordinates in turn, and then the pixel values ​​of the target image are rearranged and combined according to these corrected coordinates to construct a corrected image.

[0048] In this embodiment, the regular grid of the checkerboard image can adapt to the nonlinear distortion of the display screen, dividing the target image into sub-regions that adapt to the distortion trend. By breaking down the complex global nonlinear distortion into relatively controllable local sub-region distortions, the subsequent correction difficulty is reduced and the correction is more targeted. The coordinates of the sub-region corner points contain spatial position information and can clearly reflect the arrangement order of the sub-regions in the target image. Extracting the coordinates of the sub-region corner points helps quantify the sub-region distortion state and provides data support for the subsequent calculation of the correction matrix.

[0049] Sorting corner coordinates establishes an accurate correspondence between the subregion's corners and the standard checkerboard's corners. Sorting also corrects any confusion in the corners' spatial positions caused by nonlinear distortion, ensuring that their coordinates adhere strictly to the row and column logic of the checkerboard. This ensures that when the homography matrix is ​​subsequently calculated, the distorted corners precisely correspond to the standard corners, avoiding correction errors introduced by coordinate mismatches.

[0050] The homography matrix is ​​calculated by using the coordinates of the sub-region corner points, the coordinates of the standard black and white grid corner points, and the sorting results. The homography matrix accurately describes the conversion rules between the distorted corner points and the standard corner points. Using the calculated homography matrix to transform the target image effectively eliminates distortion, converting the distorted image into a normal, clear image that most closely resembles the actual display screen state. This enables the defect detection system to accurately identify defects such as scratches, bright spots, and dark spots on the display, improving inspection efficiency and quality.

[0051] In the above embodiment, among the extracted sub-region corner coordinates, there may be abnormal corner coordinates caused by factors such as image noise, occlusion, and local severe distortion. Figure 12 As shown, Figure 12 These are abnormal corner coordinates that appear after severe distortion. Due to severe distortion, some corner locations are blurred, resulting in multiple coordinates for the same corner. Furthermore, impurities and defects in the small white squares can easily cause interference. To ensure the accuracy and stability of subsequent calculations, it is necessary to remove these abnormal points.

[0052] See also Figure 2 , the present application provides an embodiment of a method for removing outliers, comprising: 201. Setting a row spacing threshold and a column spacing threshold between sub-regions; 202. Sort the row coordinates of the corner point coordinates and obtain the first row coordinates of the first corner point coordinates; 203. Calculate the first common row upper and lower limits based on the first row coordinates and the row spacing threshold, and obtain a row coordinate array based on the first common row upper and lower limits; 204. Extract the corresponding column coordinate array according to the row coordinate array; 205. Calculate the values ​​in the column coordinate array to obtain a column coordinate difference array; 206. If there is a column coordinate difference that is less than the column distance threshold, extract and summarize it into an outlier array; 207. Analyze the elements in the outlier array according to preset conditions, and remove corresponding outliers according to the analysis results. The elements in the outlier array include column coordinate differences that are less than a column distance threshold.

[0053] In a checkerboard image, the row and column spacing between subregions is normally relatively stable. However, factors such as image noise, occlusion, and severe local distortion can cause the extracted subregion corner coordinates to be abnormal, causing the row and column spacing to deviate from the normal range. Setting row and column spacing thresholds defines a reasonable distance range for determining which corner coordinates are likely to be outliers. These thresholds can be determined by taking the average of the row and column spacing between subregions in a large number of normal checkerboard images and considering a certain range of fluctuations.

[0054] Sorting the row coordinates of the corner points is to arrange the corner points in row order, which facilitates subsequent row-based processing. Obtaining the first row coordinate of the first corner point is to determine the starting row for processing, providing a reference for the subsequent calculation of the upper and lower limits of the common rows.

[0055] Using the first row coordinate of the first corner point as a reference and combining it with the row spacing threshold, we can calculate the row coordinate range of corner points that may belong to the same row, namely the first co-row upper and lower limits. Corner points corresponding to row coordinates within this range are likely to be located in the same row. Using this range to filter the row coordinate array is to group corner points that may be in the same row together, facilitating subsequent processing of column coordinates.

[0056] After obtaining the row coordinate arrays of possible common rows, the column coordinates corresponding to these row coordinates are extracted from the original set of corner point coordinates to form a column coordinate array. This allows the column coordinates of corner points in the same row to be grouped together, making it easier to analyze the relationship between the column coordinates and determine whether there are any outliers.

[0057] Subtract adjacent values ​​from the column coordinate array to generate an array of column coordinate differences. The column coordinate differences reflect the column distances between adjacent corner points in the same row. By analyzing these differences, you can determine whether there are corner points with unusual column distances, known as outliers.

[0058] Normally, the column distance between adjacent corner points in the same row should be greater than or equal to the column distance threshold. If the column coordinate difference is less than the column distance threshold, it indicates that the distance between the two adjacent corner points is too close, which may indicate an anomaly, such as duplicate corner points or erroneous corner points caused by noise. The corner points corresponding to these anomalies are extracted and summarized into an anomaly point array for subsequent processing.

[0059] Analyze the elements in the outlier array based on pre-set conditions to determine whether the outlier is due to duplicate extraction or noise. Based on the different judgment results, adopt corresponding strategies to remove the outlier. If the outlier is a duplicated corner point, one of them can be directly deleted. If the outlier is caused by noise, it can be corrected or deleted based on the information of surrounding corner points.

[0060] In this embodiment, removing outliers can prevent these erroneous data from interfering with the subsequent calculation of the homography transformation matrix. The row and column thresholds can be adjusted according to the actual degree of distortion, avoiding excessive removal of normal points while strictly filtering out obvious outliers.

[0061] The homography transformation matrix is ​​calculated based on the accurate coordinates of the corner points. The presence of outliers can lead to inaccurate matrix calculations, which in turn affects the image correction effect. By removing outliers, the calculated homography transformation matrix can more accurately describe the image distortion, thereby improving the accuracy of image correction and making the corrected image closer to the real scene.

[0062] See also Figure 3 , the present application provides an embodiment of a method for removing outliers based on analysis results, comprising: 301. If the element in the outlier array contains the first element of the column coordinate difference array, remove the sub-region corner point coordinates associated with the first element in the row coordinate array; The first element of the column coordinate difference array represents the difference in column distance between the first two corner points (the first and second) in the row coordinate array. If this difference value is present in the outlier array, it indicates an anomaly in the column distance between two adjacent corner points. Typically, in regular images like checkerboards, the distribution of corner points is relatively uniform and orderly. If the first difference value is anomalous, it likely indicates a problem with the first corner point itself, such as noise or an erroneous point from repeated extraction. Therefore, the subregion corner point coordinates associated with the first element in the row coordinate array are removed to eliminate the impact of this anomaly on subsequent processing.

[0063] 302. If an element in the outlier array is the last element in the column coordinate difference array, remove the sub-region corner point coordinates associated with the next element after the corresponding element in the row coordinate array; The last element of the column coordinate difference array represents the difference in column-wise distance between the last two corner points in the row coordinate array. An abnormal difference indicates that the distance between the last two corner points is outside the normal range. Given the regular arrangement of checkerboard corner points, a problem with the last corner point is more likely, possibly due to noise interference or partial occlusion at the edge of the image. Therefore, the subregion corner coordinates associated with the element following the corresponding element (the second-to-last element) in the row coordinate array are removed to correct the anomaly.

[0064] 303. If the element in the outlier array is neither the first element nor the last element in the column coordinate difference array, obtain the first difference between the corresponding element and the previous element in the row coordinate array; 304. If the first difference is less than the column distance threshold, remove the sub-region corner coordinates associated with the corresponding element in the row coordinate array; otherwise, remove the sub-region corner coordinates associated with the next element after the corresponding element in the row coordinate array.

[0065] When an element in the outlier array is in the middle of the column coordinate difference array, it indicates that the column distances between adjacent corner points in the middle of the row coordinate array are abnormal. This is determined by calculating the first difference between the corresponding element in the row coordinate array (i.e., the row coordinate where the outlier occurs) and the previous element in the row coordinate array, and by combining the distribution patterns of the checkerboard corner points in the row and column directions.

[0066] If the first difference is less than the column distance threshold, the corresponding element in the row coordinate array (the row coordinate where the abnormal difference occurs) is relatively close to the previous element in the row direction. Considering the uniformity of the checkerboard corner distribution, it's more likely that the corner associated with the corresponding element has a problem. This corner may be a noise point or misplaced due to local distortion, so the coordinates of that corner point are removed.

[0067] When the first difference is greater than or equal to the column distance threshold, it indicates that the row distance between the corresponding element and the previous element in the row coordinate array is relatively normal. In this case, an abnormal column distance is more likely due to a problem with the corner point associated with the element following the corresponding element, perhaps due to excessive offset in the column direction. Therefore, the subregion corner point coordinates associated with the element following the previous element are removed.

[0068] In this embodiment, the above steps accurately identify and remove outliers, significantly improving the accuracy of corner point coordinate data. Differentiation rules are established for the "first, last, and middle" positions of outliers in the column coordinate difference array to prevent the accidental deletion of normal points. After outlier removal, the corner point coordinates show a significantly reduced proportion of outliers, and the retained corner point sequence is continuous and has consistent spacing.

[0069] Accurate corner point coordinates provide precise correspondence between distorted corner points and standard corner points for subsequent homography transformation matrix calculation, reducing matrix parameter deviations caused by abnormal points, and helping to improve the quality and effectiveness of the entire image processing process.

[0070] See also Figure 4 , the present application provides an embodiment of a method for sorting sub-region corner point coordinates, comprising: Step 1: Sort the row coordinates in the row coordinate array and the column coordinates in the column coordinate array, and extract the minimum row coordinate and the minimum column coordinate; Step 2: Obtain the upper and lower limits of the same row and column corresponding to the minimum row coordinate and the minimum column coordinate according to the row spacing threshold and the column spacing threshold; Step 3: Record and store the row coordinates and column coordinates that meet the common row and column upper and lower limits into the common row index set and the common column index set respectively; Step 4: Remove the row coordinates and column coordinates in the row coordinate array and the column coordinate array that have been stored in the common row index set and the common column index set; Step 5: Repeat steps 1 to 4 until all row coordinates and column coordinates in the row coordinate array and column coordinate array are traversed; Step 6: Traverse all common row index sets and common column index sets, and extract the index values ​​that appear in both common row index sets and common column index sets; Step 7: Based on the co-row index set and co-column index set where the index value is located, put the sub-region corner point coordinates corresponding to the index value into a new coordinate set. The length of the new coordinate set is the product of the number of rows and columns of black and white squares in the checkerboard image. Step 8: Repeat steps 6 and 7 until all sub-region corner point coordinates corresponding to the index values ​​are placed in the new coordinate set.

[0071] Sort the row and column coordinate arrays separately, extracting the minimum row coordinate (the reference for the top row of the checkerboard) and the minimum column coordinate (the reference for the leftmost column). These minimum row and column coordinates are the starting points for sorting the checkerboard, ensuring order is established starting from the top left corner of the checkerboard.

[0072] Based on the row and column spacing thresholds, the upper and lower limits of the minimum row coordinate and the upper and lower limits of the minimum column coordinate are calculated. The upper and lower limits of the minimum row coordinate are the minimum row coordinate ± the row spacing threshold, corresponding to the normal row coordinate range of the minimum row. The upper and lower limits of the minimum column coordinate are the minimum column coordinate ± the column spacing threshold, corresponding to the normal column coordinate range of the minimum column. This provides a judgment standard for screening corner points in the same row and column.

[0073] The indices of the corner points whose row coordinates are within the upper and lower limits of the same row are stored in the same row index set (corner point indices of the same row), and the indices of the corner points whose column coordinates are within the upper and lower limits of the same column are stored in the same column index set (corner point indices of the same column).

[0074] To avoid repeated processing of corner points whose positions have already been determined, the row and column coordinates that have already been recorded in the index set are removed from the original array. This can gradually narrow the processing scope and improve the efficiency of the algorithm.

[0075] By repeatedly repeating the above steps, the positions of all corner points on the chessboard are gradually found and recorded in the corresponding index set. Each repetition uses the minimum row and column coordinates of the remaining corner points as a benchmark to determine new upper and lower limits for co-row and co-column co-coordinates, filtering out corner points that meet the criteria. This is equivalent to a "row and column scan," first filtering all corner points in the first row, then the second row, until all rows and columns of the chessboard are covered.

[0076] Traverse all common row index sets (corner indexes for each row) and column index sets (corner indexes for each column) and extract the index values ​​that appear in both the row and column sets. The corner points corresponding to the index values ​​that appear in both sets are the actual corner points on the chessboard. By traversing all sets and extracting these index values, you can determine the valid corner point locations.

[0077] Based on the extracted index value, the corresponding sub-region corner coordinates are obtained from the original row and column coordinate arrays. These coordinates are placed in a new coordinate set in a certain order. The length of the new coordinate set is the product of the number of rows and columns of black and white squares in the checkerboard image. This ensures that the new coordinate set can completely store all valid corner coordinates and is arranged according to the pattern of the checkerboard.

[0078] Repeat the matching process until all corner point coordinates are filled into the new coordinate set according to row and column positions, and the sorting is completed.

[0079] In this embodiment, traditional sorting can lead to the inadvertent deletion of severely distorted corner points due to their deviation from the visual rows and columns. However, the "periodic extraction of minimum coordinates followed by cyclic screening of remaining coordinates" approach accurately locates all valid corner coordinates within the checkerboard, ensuring that all corner points (including those with edge distortion) fall into corresponding co-row or co-column sets. These points are then assigned to a new coordinate set through index matching. In this sorted new coordinate set, the coordinates of each corner point at each position clearly correspond to the standard corner points on the checkerboard, reducing the processing complexity of subsequent steps.

[0080] like Figure 13 As shown, Figure 13 This is a diagram of unextracted corner points. Due to factors such as image blur or insufficient local contrast, some unextracted corner points are marked as -1 in the sorted coordinate set. To subsequently map all corner point coordinates to the corrected image, the coordinates of these missing corner points must be calculated.

[0081] See also Figure 5 , the present application provides an embodiment of a method for supplementing missing corner point coordinates, comprising: 501. Determine whether the number of co-row index sets or co-column index sets is less than the number of rows and columns of black and white squares in the checkerboard image; 502. If yes, calculate the missing corner coordinates based on the common row index set and the common column index set; 503. Put the missing corner point coordinates into the new coordinate set.

[0082] The common row index set records the column index information of each row in the checkerboard image where the corner points have been successfully extracted. The common column index set records the row index information of each column where the corner points have been successfully extracted.

[0083] The purpose of determining whether the number of co-row index sets or co-column index sets is less than the number of rows and columns of black and white squares in the checkerboard image is to determine whether there are missing corner rows or columns in the checkerboard image. Ideally, the number of co-row index sets and co-column index sets should be equal to the actual number of rows and columns of black and white squares in the checkerboard image. By comparing the number of co-row index sets or co-column index sets with the number of rows and columns of black and white squares in the checkerboard image, if the number is less than the actual number of rows and columns, it indicates that there are missing corner rows or columns.

[0084] After determining that there are missing corner rows or columns, the coordinates of the missing corners are calculated using the existing co-row index set and co-column index set. Since the checkerboard has a certain regularity, its corners usually show a regular grid distribution in the image.

[0085] For rows with missing corner points, use the coordinates of the extracted corner points in that row and employ an appropriate fitting method (such as the least squares method) to fit the distribution curve of the corner points in that row. Assuming the coordinates of the corner points in a row have been extracted, the least squares method can be used to find a straight line that minimizes the sum of the squared errors between the corner points and the straight line. Similarly, for columns with missing corner points, use the coordinates of the extracted corner points in that column to fit the distribution curve of the corner points in that column.

[0086] To more accurately obtain the coordinates of the missing corner points, we can consider the fitting results of both co-row and co-column. Specifically, we can first perform co-row fitting to obtain the distribution pattern of the corner points in each row, then perform co-column fitting to obtain the distribution pattern of the corner points in each column, and then use a certain algorithm to determine the coordinates of the missing corner points.

[0087] The coordinates of the missing corner points can be calculated by fitting the co-row and co-column curve equations and combining them with the regularity of the checkerboard. For example, in a regular checkerboard, the spacing between adjacent corner points is known to be fixed. After determining the approximate location of the missing corner point using the fitted curve, this spacing information can be used to accurately calculate the coordinates of the missing corner point.

[0088] Finally, the calculated missing corner point coordinates are put into the new coordinate set in a certain format and order.

[0089] In this embodiment, in order to avoid missing corner points due to unclear images and insufficient contrast, the missing coordinates are calculated by fitting a curve to avoid the occurrence of unextracted corner points in the new coordinate set.

[0090] Nonlinear distortion has spatial continuity. The co-row and co-column fitting curve is based on spatial continuity and uses the distribution trend of the existing corner point coordinates to infer the missing values. This ensures that the completed corner point coordinates conform to the overall distortion law and have a smooth transition with the coordinates of adjacent valid corner points, avoiding coordinate distortion caused by arbitrary filling.

[0091] Calculating the homography matrix requires a one-to-one correspondence between the target image's corner coordinates and the checkerboard's standard corner coordinates (equal in number and position). After completing missing coordinates, the new coordinate set contains exactly the same number of corners as the standard corners, and each completed coordinate corresponds to the correct position, ensuring a flawless match. This provides complete coordinate pairs for matrix calculation, reduces matrix parameter errors caused by missing data, and improves the accuracy of subsequent image correction.

[0092] See also Figure 6 The present application provides an embodiment of a method for calculating the coordinates of missing corner points, including: 601. Traverse the new coordinate set and record the row and column numbers of the missing corner coordinates; 602. Extract the co-row index set and the co-column index set of the missing corner point coordinates according to the row and column numbers. 603. Calculate the co-row second-order fitting coefficient and the co-column second-order fitting coefficient according to the co-row index set and the co-column index set; 604. Calculate the coordinates of the missing corner points based on the co-row second-order fitting coefficients and the co-column second-order fitting coefficients.

[0093] Visit each element in the new coordinate set in turn. For each element, check whether it is the identifier of the missing corner point. If so, record the row and column number of the missing corner point.

[0094] Based on the row and column numbers of the missing corner point, extract the corresponding co-row index set (all valid corner point coordinates in the row where the missing point is located) and co-column index set (all valid corner point coordinates in the column where the missing point is located). These sets contain valid coordinates in the same row or column as the missing point, and their distribution pattern can reflect the coordinate trend of the missing point.

[0095] For the valid coordinates in the co-row index set, the second-order fitting coefficient A of the co-row corner points and the second-order fitting coefficient A' of the co-column corner points are calculated by polynomial fitting. The second-order fitting coefficient A of the co-row corner points describes the nonlinear law that the column coordinates in the same row change with the column number, and the second-order fitting coefficient A' describes the nonlinear law that the row coordinates in the same column change with the row number.

[0096] Polynomial fitting formula:

[0097] During the polynomial fitting process, the positions of X and Y need to be swapped, that is, Y is used as the independent variable, and the second-order fitting coefficient A' of the co-ranked corner points is calculated.

[0098] The polynomial fitting formula shows that the highest order of the independent variable in the fitted polynomials Y = A·X and X = A'·Y, which share common rows and columns, is 2nd. Calculating the intersection of these two equations using a system of simultaneous equations would produce a 4th-order independent variable. However, since there are only two equations in the system, the X and Y values ​​at the intersection cannot be calculated.

[0099] 4th-order independent variable formula:

[0100] Among them, a, b, c, d, and e are coefficients; In order to calculate the coordinates of the intersection, the line drawing method is used to find the intersection: The independent variable set is generated by setting the step size based on the length of the checkerboard squares. For example, assuming the horizontal length of the checkerboard squares (corresponding to the relative position of the corner points in the row direction) is L, starting from one end of the row where the missing corner point is located, the value of the independent variable x is gradually increased by a step size Δx (generally less than L, such as L / 10) until the other end of the row where the missing corner point is located is covered, thus generating a row-wise independent variable set. Similarly, for the column-wise second-order polynomial, the step size is set based on the vertical length of the checkerboard squares (corresponding to the relative position of the corner points in the column direction) to generate a column-wise independent variable set.

[0101] Substitute each independent variable x in the row-wise independent variable set into the row-wise polynomial to calculate the corresponding row coordinate, thus obtaining the row-wise coordinate set. Substitute each independent variable y in the column-wise independent variable set into the column-wise polynomial to calculate the corresponding column coordinate, thus obtaining the column-wise coordinate set.

[0102] Two parabolas are drawn based on the coordinate sets calculated in the row and column directions. The row coordinate set corresponds to a parabola on the checkerboard image plane, which reflects the distribution trend of the corner points in the row direction; the column coordinate set corresponds to another parabola, which reflects the distribution trend of the corner points in the column direction.

[0103] Calculate the intersection points of these two parabolas with the four edges of the image respectively. For each parabola, solve the system of simultaneous equations of the parabola equation and the equation of the line on which the edge of the image lies to obtain the coordinates of the intersection point. Figure 14 As shown, the intersection serves as the starting and ending points of the parabola. Find the intersection of the two parabolas on the image plane. This intersection is the common portion of the range covered by the two parabolas. It represents the area that satisfies the corner point distribution trends in both the row and column directions after fitting a second-order polynomial in the row and column directions.

[0104] Perform coordinate statistics on all pixels in the intersection area and calculate the average coordinates of these pixels in the x and y directions, which are the coordinates of the missing corner points. Figure 15 As shown, Figure 15 A schematic diagram for filling in the missing corner coordinates.

[0105] In this embodiment, due to factors such as lens distortion and image perspective, the distribution of corner points in an actual checkerboard image is often not strictly linear, but rather exhibits a certain degree of curvature. Second-order fitting can more accurately describe this curvature, thereby calculating more precise coordinates of missing corner points. By simultaneously considering the second-order fitting information for both co-row and co-column locations, the coordinates of missing corner points are solved using simultaneous equations. This fully exploits the correlation between corner points on the checkerboard in both row and column directions, reduces errors introduced by fitting in a single direction, and improves the accuracy of coordinate calculation.

[0106] See also Figure 7 The present application provides an embodiment of a device for correcting nonlinear image distortion, comprising: The segmentation unit 701 is used to input the checkerboard image into the target image and segment the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that produces nonlinear distortion; An extraction unit 702 is used to extract the coordinates of the sub-region corner points of the sub-region; A sorting unit 703 is used to sort the coordinates of the sub-region corner points; A calculation unit 704 is configured to calculate a homography transformation matrix based on the sub-region corner point coordinates, the black and white grid corner point coordinates of the checkerboard image, and the sorting result, where the black and white grid corner point coordinates are standard corner point coordinates of a plurality of black and white grids in the checkerboard image; The conversion unit 705 is configured to convert the target image into a rectified image using a homography transformation matrix.

[0107] Optionally, before the sorting unit 703, the correction device further includes: The removal unit 706 is configured to remove abnormal points in the coordinates of the sub-region corner points.

[0108] Specific implementation method Figures 1 to 6 The embodiments are not described in detail here.

[0109] See also Figure 8 , the present application also provides a device for correcting nonlinear image distortion, comprising: Processor 801 , memory 802 , input / output unit 803 , and bus 804 .

[0110] The processor 801 is connected to the memory 802 , the input and output unit 803 , and the bus 804 .

[0111] The memory 802 stores a program, and the processor 801 calls the program to execute the following Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 and Figure 6 Correction method in .

[0112] The present application provides a computer-readable storage medium, wherein a program is stored on the computer-readable storage medium, and when the program is executed on a computer, the program performs the following operations: Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 and Figure 6 Correction method in .

[0113] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0115] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0116] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A method for correcting nonlinear image distortion, characterized in that: include: Inputting a checkerboard image into a target image, and dividing the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that produces nonlinear distortion; Extracting the coordinates of the sub-region corner points of the sub-region; Sorting the coordinates of the sub-region corner points; Calculating a homography transformation matrix according to the sub-region corner point coordinates, the black and white grid corner point coordinates of the checkerboard image, and the sorting result, wherein the black and white grid corner point coordinates are standard corner point coordinates of a plurality of black and white grids in the checkerboard image; The target image is converted into a rectified image by using the homography transformation matrix.

2. The correction method according to claim 1, characterized in that: Before sorting the coordinates of the sub-region corner points, the correction method further includes: Remove outliers in the coordinates of the sub-region corner points.

3. The correction method according to claim 2, characterized in that: The removing of abnormal points in the coordinates of the sub-region corner points includes: Set the row and column spacing thresholds between sub-regions; Sort the row coordinates of the corner point coordinates, and obtain the first row coordinate of the first corner point coordinate; Calculating first common row upper and lower limits according to the first row coordinates and the row spacing threshold, and obtaining a row coordinate array according to the first common row upper and lower limits; According to the row coordinate array, extract the corresponding column coordinate array; Calculating the values ​​in the column coordinate array to obtain a column coordinate difference array; If there is a column coordinate difference that is less than the column distance threshold, extract and summarize it into an abnormal point array; The elements in the outlier array are analyzed according to preset conditions, and corresponding outliers are removed according to the analysis results, wherein the elements in the outlier array include column coordinate differences that are less than the column distance threshold.

4. The correction method according to claim 3, characterized in that: Analyzing the elements in the outlier array according to preset conditions and removing corresponding outliers according to the analysis results includes: If the first element of the column coordinate difference array exists in the element of the outlier array, then the sub-region corner point coordinates associated with the first element in the row coordinate array are removed; If the element in the outlier array contains the last element of the column coordinate difference array, then the sub-region corner coordinates associated with the next element after the corresponding element in the row coordinate array are removed; If the element in the outlier array is neither the first element nor the last element in the column coordinate difference array, obtaining the first difference between the corresponding element and the previous element in the row coordinate array; If the first difference is less than the column distance threshold, the sub-region corner coordinates associated with the corresponding element in the row coordinate array are removed; otherwise, the sub-region corner coordinates associated with the next element after the corresponding element in the row coordinate array are removed.

5. The correction method according to claim 3 or 4, characterized in that: The sorting of the sub-region corner point coordinates includes: Step 1: sorting the row coordinates in the row coordinate array and the column coordinates in the column coordinate array, and extracting the minimum row coordinate and the minimum column coordinate; Step 2: Obtaining the upper and lower limits of the same row and the same column corresponding to the minimum row coordinate and the minimum column coordinate according to the row spacing threshold and the column spacing threshold; Step 3: Record and store the row coordinates and column coordinates that satisfy the common row upper and lower limits and the common column upper and lower limits into a common row index set and a common column index set respectively; Step 4: removing the row coordinates and the column coordinates in the row coordinate array and the column coordinate array that have been stored in the co-row index set and the co-column index set; Step 5: Repeat steps 1 to 4 until all the row coordinates and column coordinates in the row coordinate array and the column coordinate array are traversed; Step 6: Traverse all common row index sets and common column index sets, and extract the index values ​​that appear in both common row index sets and common column index sets; Step 7: Based on the co-row index set and the co-column index set where the index value is located, the sub-region corner point coordinates corresponding to the index value are placed into a new coordinate set, wherein the length of the new coordinate set is the product of the number of rows and columns of black and white squares in the checkerboard image; Step 8: Repeat steps 6 and 7 until all sub-region corner point coordinates corresponding to the index values ​​are put into the new coordinate set.

6. The correction method according to claim 5, characterized in that: Before step eight, the correction method further includes: Determine whether the number of the co-row index set or the co-column index set is less than the number of rows and columns of black and white squares in the checkerboard image; If yes, the coordinates of the missing corner points are calculated based on the co-row index set and the co-column index set; The missing corner point coordinates are put into the new coordinate set.

7. The correction method according to claim 6, characterized in that: The calculating the missing corner point coordinates according to the co-row index set and the co-column index set includes: Traverse the new coordinate set and record the row and column numbers of the missing corner point coordinates; According to the row and column numbers, respectively extracting the co-row index set and the co-column index set where the coordinates of the missing corner point are located; Calculating a co-row second-order fitting coefficient and a co-column second-order fitting coefficient according to the co-row index set and the co-column index set; The coordinates of the missing corner points are calculated based on the co-row second-order fitting coefficients and the co-column second-order fitting coefficients.

8. A device for correcting nonlinear image distortion, characterized in that: include: a segmentation unit, configured to input a checkerboard image into a target image and segment the target image into a plurality of sub-regions according to the checkerboard image, wherein the target image is an image that generates nonlinear distortion; An extraction unit, configured to extract coordinates of sub-region corner points of the sub-region; A sorting unit, configured to sort the coordinates of the sub-region corner points; a calculation unit, configured to calculate a homography transformation matrix based on the sub-region corner point coordinates, the black and white grid corner point coordinates of the checkerboard image, and the sorting result, wherein the black and white grid corner point coordinates are standard corner point coordinates of a plurality of black and white grids in the checkerboard image; A conversion unit is configured to convert the target image into a rectified image using the homography transformation matrix.

9. A device for correcting nonlinear image distortion, characterized in that: The device comprises: processor, memory, input and output units, and buses; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the correction method according to any one of claims 1 to 7 is executed.

Images