An image distortion correction method, device, equipment and storage medium

Abstract

The disclosure provides an image distortion correction method and device, equipment and storage medium, relating to the technical field of image processing. The method mainly comprises: acquiring multiple calibration plate images and parameter information of the calibration plate images; performing corner point extraction on the calibration plate images to obtain target corner points; establishing a virtual standard grid corresponding to the calibration plate images according to the parameter information and the target corner points; calculating a first offset between the target corner points and the vertices of the virtual standard grid corresponding to the positions of the target corner points to obtain a corner point offset vector diagram corresponding to the calibration plate images; performing inverse linear interpolation on a preset table corresponding to the calibration plate images according to the multiple corner point offset vector diagrams to obtain a distortion table; and performing distortion correction on a to-be-corrected image according to the distortion table to obtain an image distortion correction result.

Description

A method, apparatus, device and storage medium for image distortion correction Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image distortion correction method, apparatus, device, and storage medium. Background Technology

[0002] With the development of computer vision, various industries have a need to acquire accurate image data. For example, in the industrial field, accurate workpiece image data is needed; in the transportation field, accurate road image data is needed. However, due to the shooting angle or the inherent errors of image acquisition equipment such as cameras, the acquired images are often distorted.

[0003] In existing technologies, Zhang's calibration algorithm is typically used to correct image distortion. This involves establishing an imaging model based on the mapping relationship between the physical scene and its corresponding image, calculating the intrinsic and extrinsic parameters of the image acquisition device, and then using these parameters to correct image distortion. However, Zhang's calibration algorithm cannot meet the high accuracy requirements of scenarios such as image measuring instruments, and it is not suitable for image acquisition devices with large depth of field and varying extrinsic parameters. Summary of the Invention

[0004] This disclosure provides an image distortion correction method, apparatus, device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this disclosure, an image distortion correction method is provided, the method comprising: acquiring multiple calibration plate images and parameter information of the calibration plate images; extracting corner points from the calibration plate images to obtain target corner points; establishing a virtual standard grid corresponding to the calibration plate images based on the parameter information and the target corner points; calculating a first offset between the target corner point and the vertex of the virtual standard grid corresponding to its position to obtain a corner point offset vector map corresponding to the calibration plate images; performing inverse linear interpolation on a preset table corresponding to the calibration plate images based on the multiple corner point offset vector maps to obtain a distortion table; and performing distortion correction on the image to be corrected based on the distortion table to obtain an image distortion correction result.

[0006] According to a second aspect of this disclosure, an image distortion correction device is provided, comprising: an acquisition module for acquiring multiple calibration plate images and parameter information of the calibration plate images; a corner extraction module for extracting corners from the calibration plate images to obtain target corners; an establishment module for establishing a virtual standard grid corresponding to the calibration plate images based on the parameter information and the target corners; a calculation module for calculating a first offset between the target corners and the vertices of the virtual standard grid corresponding to their positions to obtain a corner offset vector map corresponding to the calibration plate images; a distortion table generation module for performing inverse linear interpolation on a preset table corresponding to the calibration plate images based on the multiple corner offset vector maps to obtain a distortion table; and a distortion correction module for performing distortion correction on the image to be corrected based on the distortion table to obtain an image distortion correction result.

[0007] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0008] At least one processor; and

[0009] A memory communicatively connected to the at least one processor; wherein,

[0010] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0011] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0012] This disclosure discloses an image distortion correction method, apparatus, device, and storage medium. Based on the parameter information and target corner points of a calibration board image, a virtual standard grid corresponding to the calibration board image is established. Based on the first offset of the target corner points relative to the vertices of the virtual standard grid, a corner point offset vector map corresponding to the calibration board image is determined. Then, based on multiple corner point offset vector maps, a preset table corresponding to the calibration board image is subjected to inverse linear interpolation to obtain a distortion table. Distortion correction is then performed on the image to be corrected based on the distortion table to obtain the image distortion correction result. Therefore, this method, based on the characteristics of image acquisition devices such as cameras and lenses, uses a non-parametric calibration method to calibrate the distortion table. Thus, it is suitable for image acquisition devices with large depth of field and varying extrinsic parameters, and by performing distortion correction on the image to be corrected based on the distortion table, higher accuracy can be achieved.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0015] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0016] Figure 1 shows a flowchart of an image distortion correction method according to a first embodiment of the present disclosure;

[0017] Figure 2 shows a schematic diagram of a first scenario of an image distortion correction method according to a first embodiment of the present disclosure;

[0018] Figure 3 shows a second scenario schematic diagram of an image distortion correction method according to the first embodiment of this disclosure;

[0019] Figure 4 shows a distortion table according to a first embodiment of this disclosure;

[0020] Figure 5 shows a flowchart of an image distortion correction method according to a second embodiment of the present disclosure;

[0021] Figure 6 shows a schematic diagram of a first scenario of an image distortion correction method according to a second embodiment of the present disclosure;

[0022] Figure 7 shows a second scenario schematic diagram of an image distortion correction method according to a second embodiment of the present disclosure;

[0023] Figure 8 shows a schematic diagram of a third scenario of an image distortion correction method according to a second embodiment of the present disclosure;

[0024] Figure 9 shows a flowchart of an image distortion correction method according to a third embodiment of the present disclosure;

[0025] Figure 10 shows a schematic diagram of a scenario for an image distortion correction method according to a third embodiment of the present disclosure;

[0026] Figure 11 shows a schematic diagram of a scenario for an image distortion correction method according to the fourth embodiment of this disclosure;

[0027] Figure 12 shows a schematic diagram of a first scenario of an image distortion correction method according to the fifth embodiment of this disclosure;

[0028] Figure 13 shows a second scenario schematic diagram of an image distortion correction method according to the fifth embodiment of this disclosure;

[0029] Figure 14 shows a flowchart of an image distortion correction method according to a sixth embodiment of the present disclosure;

[0030] Figure 15 shows a schematic diagram of a scenario for an image distortion correction method according to the seventh embodiment of this disclosure;

[0031] Figure 16 shows a schematic diagram of the structure of an image distortion correction device according to the eighth embodiment of the present disclosure;

[0032] Figure 17 shows a schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0033] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0034] Figure 1 shows a flowchart of an image distortion correction method according to a first embodiment of this disclosure. As shown in Figure 1, the image distortion correction method mainly includes:

[0035] Step S101: Obtain multiple calibration plate images and parameter information of the calibration plate images.

[0036] In this embodiment, the calibration board is a geometric model used to determine lens distortion and the conversion relationship between physical dimensions and pixels in applications such as machine vision, image measurement, and 3D reconstruction. The calibration board can be a checkerboard calibration board or a dot calibration board, etc., and this disclosure does not limit the style of the calibration board. Multiple calibration board images are multiple images taken of the calibration board. For example, after setting up the image acquisition device, the same calibration board can be translated or rotated to obtain multiple different calibration board images. The parameter information of the calibration board images includes the pixel size, rotation angle, and pixel equivalent of the calibration board images. Figure 2 shows a checkerboard calibration board and a dot calibration board. For ease of display, the checkerboard calibration board images in subsequent embodiments will be displayed as a table without black and white fill.

[0037] Step S102: Extract corner points from the calibration board image to obtain the target corner points.

[0038] In this embodiment, corner points are reference points used for calibration in the calibration board image. For example, in a checkerboard calibration board image, corner points are the vertices of each cell in the checkerboard, and in a dot calibration board image, corner points are the dots of each dot. For a checkerboard calibration board image, a corner detection algorithm can be used to extract the vertices of each cell in the calibration board image to obtain the target corner points. Corner detection algorithms can include the Harris corner detection algorithm and the Shi-Tomasi corner detection algorithm, etc. For a dot calibration board image, the findCirclesGrid function in OpenCV can be used to extract the coordinates of the center of each circle to obtain the target corner points. OpenCV is an open-source computer vision library.

[0039] Step S103: Based on the parameter information and target corner points, establish a virtual standard grid corresponding to the calibration board image.

[0040] In this embodiment, the parameter information includes the rotation angle of the calibration board image and the actual pixel size of the calibration unit in the calibration board image. The rotation angle is the rotation angle of the calibration board image relative to the image acquisition device such as a camera. The calibration unit is the smallest unit in the calibration board. For example, the calibration unit of a checkerboard calibration board is each grid, so the actual pixel size can be the actual pixel width and actual pixel height of one grid. The calibration unit of a dot calibration board is each dot, so the actual pixel size can be the diameter of the dot or the distance between the centers of two adjacent dots.

[0041] In this embodiment, when establishing the virtual standard grid corresponding to the calibration board image, a target corner point can be selected. Using this target corner point as the center, and the actual pixel size of the calibration unit as the actual pixel size of the sub-grids of the virtual standard grid, each vertex of the virtual standard grid is determined along the direction of the rotation angle until the area enclosed by the vertices covers all target corner points, thus obtaining a distortion-free virtual standard grid. Generally, the distortion is smaller at the center of the calibration board image; therefore, the selected target corner point can be the one closest to the center of the calibration board image.

[0042] Step S104: Calculate the first offset between the target corner point and the virtual standard grid vertex corresponding to its position to obtain the corner offset vector map corresponding to the calibration board image.

[0043] In this embodiment, all target corner points in the calibration board image have corresponding virtual standard grid vertices in the virtual standard grid. The first offset between all target corner points and their corresponding virtual standard grid vertices is calculated, and the first offset is marked on the corresponding target corner point to obtain the corner point offset vector map corresponding to the calibration board image.

[0044] Figure 3 shows a second scenario schematic diagram of an image distortion correction method according to the first embodiment of this disclosure. As shown in Figure 3, taking a checkerboard calibration board image as an example, the dashed table in Figure 3 represents the checkerboard corresponding to the checkerboard calibration board image, which has a certain distortion. The solid table in Figure 3 represents the virtual standard grid corresponding to the checkerboard in the checkerboard calibration board image. For a target corner point A in the checkerboard calibration board image, the vertex of the virtual standard grid corresponding to its position is point B. The first offset of target corner point A relative to point B can be calculated based on the coordinates of target corner point A and point B, including the first offset on the horizontal axis and the first offset on the vertical axis. From the first offset between all target corner points in the checkerboard calibration board image and the vertex of the virtual standard grid corresponding to their positions, a corner point offset vector map corresponding to the calibration board image can be generated. The corner point offset vector map not only contains the first offset corresponding to each target corner point, but also the coordinates of each target corner point. The process of calculating the first offset for the dot calibration board image is similar to that for the checkerboard calibration board image, and will not be described again here.

[0045] Step S105: Based on multiple corner offset vector maps, perform reverse linear interpolation on the preset table corresponding to the calibration plate image to obtain the distortion table.

[0046] In this embodiment, the preset table corresponding to the calibration board image is a table that is set in advance according to the size of the calibration board image. For example, according to the size of the calibration board image, the calibration board image is divided into n equal parts in both the horizontal and vertical axes to generate an n×n preset table. The value range of n can be set according to the actual situation. Preferably, the value of n can be 20.

[0047] In this embodiment, multiple corner offset vector maps are generated based on the multiple calibration board images obtained in step S101. It is understood that the corner offset vector maps, calibration board images, and preset tables are of equal size. Therefore, the multiple corner offset vector maps and the preset table can be superimposed. Using the offsets of multiple target corner points falling into each sub-table of the preset table, inverse linear interpolation is performed on each sub-table to calculate the offsets of the four vertices of each sub-table. After obtaining the offsets of all vertices of the preset table, the preset table labeled with all vertex offsets can be used as a distortion table. This is equivalent to converging the offsets and trends of countless target corner points from multiple calibration board images into the distortion table. Figure 4 shows the distortion table of the first embodiment of this disclosure. As shown in Figure 4, the points in the area enclosed by two straight lines and two arcs are target corner points, and the points outside this area are the vertices corresponding to the distortion table. Arrows indicate the offset direction of the vertices.

[0048] Step S106: Based on the distortion table, perform distortion correction on the image to be corrected to obtain the image distortion correction result.

[0049] In this embodiment, the distortion table can characterize the distortion trend of the image under the current image acquisition device. The image to be corrected is the image that needs to be distorted and captured by the current image acquisition device. Therefore, the offset of each pixel in the image to be corrected can be calculated by forward linear interpolation according to the distortion table, and the distortion correction of the image to be corrected can be performed according to the offset of each pixel in the image to be corrected to obtain the image distortion correction result.

[0050] In the first embodiment of this disclosure, based on the characteristics of image acquisition devices such as cameras and lenses, a non-parametric calibration method is used to calibrate the distortion table. Therefore, it is suitable for image acquisition devices with large depth of field and varying external parameters. Furthermore, by performing distortion correction on the image to be corrected according to the distortion table, higher accuracy requirements can be achieved.

[0051] Figure 5 shows a flowchart of an image distortion correction method according to a second embodiment of the present disclosure. As shown in Figure 5, in step S101, parameter information of the calibration plate image can be obtained in the following manner:

[0052] Step S201: Obtain the physical dimensions of the calibration units in the calibration board and the estimated pixel dimensions of the calibration units in the calibration board image.

[0053] In this embodiment, the physical size refers to the precise physical size of the calibration unit in the actual calibration board, such as the precise physical width and height of a cell in a checkerboard calibration board, or the precise physical diameter of a dot in a dot calibration board, or the physical center-to-center distance between two adjacent dots. The estimated pixel size refers to the estimated pixel size of the calibration unit in the calibration board image, such as the estimated pixel width and height of a cell in a checkerboard calibration board image, or the estimated pixel diameter of a dot in a dot calibration board image, or the estimated pixel distance between the centers of two adjacent dots. Specifically, the estimated pixel size can be obtained by delineating the line segments corresponding to the calibration unit in the calibration board image and calculating the length of the line segments. For example, in a checkerboard calibration board image, the length of the line segment between the lower left and lower right vertices of a selected cell can be used as the estimated pixel width of the calibration unit.

[0054] Step S202: Calculate the actual pixel size of the calibration unit in the calibration board image based on the physical size.

[0055] In this embodiment, the process of calculating the actual pixel size of the calibration unit in the calibration board image includes:

[0056] Step S202a: Determine the overall physical dimensions of the preset number of calibration units in the calibration plate based on the physical dimensions.

[0057] Specifically, based on the physical dimensions of the calibration units in the calibration board, the overall physical dimensions of a preset number of calibration units in the calibration board are determined. That is, a preset number of calibration units are selected in the calibration board, and their overall physical dimensions are determined by their physical dimensions. Taking a checkerboard calibration board as an example, if g adjacent left and right squares are selected, and the precise physical width of one square is widthworld, then the overall physical dimension is the overall physical width of the selected g squares, g × widthworld; if g adjacent top and bottom squares are selected, and the precise physical height of one square is heightworld, then the overall physical dimension is the overall physical height of the selected g squares, g × heightworld. Taking a dot calibration board as an example, if g adjacent dots are selected, and the physical center distance between two adjacent dots is centerdistance, then the overall physical dimension of the selected g dots is (g-1) × centerdistance.

[0058] Step S202b: Calculate the overall pixel size of the preset number of calibration units in the calibration plate image.

[0059] Specifically, the overall pixel size of a preset number of calibration units in the calibration board image is calculated, that is, the overall pixel size of the selected preset number of calibration units in the calibration board image is calculated. Taking a checkerboard calibration board image as an example, if g adjacent left and right squares are selected, the overall pixel size is the overall pixel width of the selected g squares; if g adjacent top and bottom squares are selected, the overall pixel size is the overall pixel height of the selected g squares. Taking a dot calibration board as an example, if g adjacent dots are selected, the overall pixel size is the pixel center distance between the selected 1st dot and the gth dot. The overall pixel size can be calculated based on the pixel coordinates of the center of the 1st dot and the pixel coordinates of the center of the gth dot.

[0060] Figure 6 shows a schematic diagram of a first scenario of an image distortion correction method according to the second embodiment of this disclosure. As shown in Figure 6, taking a checkerboard calibration board image as an example, if a, b, c, d, and e are five selected adjacent grids, then the overall pixel width of these five grids is the distance between the lower left vertex C of grid a and the lower right vertex D of grid e. The overall pixel width of these five grids can be calculated based on the pixel coordinates of points C and D. The calculation method for the overall pixel height in the checkerboard calibration board image is similar to the calculation method for the overall pixel width, and will not be repeated here.

[0061] Step S202c: Calculate the pixel equivalent of the calibration board image based on the overall physical size and overall pixel size.

[0062] Step S202d: Calculate the actual pixel size of the calibration unit in the calibration board image based on the physical size and pixel equivalent.

[0063] Specifically, the pixel equivalent of the calibration board image can be calculated using the following formula: Among them, td world For the overall physical dimensions, td pixel This refers to the overall pixel size.

[0064] Specifically, the actual pixel size of the calibration unit in the calibration plate image can be calculated using the following formula. Where, d world Where d is the physical size, and PixelEquivalent is the pixel equivalent. It's important to emphasize that, taking a checkerboard calibration board image as an example, if we calculate the actual pixel width within the actual pixel size of the calibration unit, then d... world Let d be the physical width of the calibration unit in the calibration board; if we calculate the actual pixel height in the actual pixel size of the calibration unit, then d world Let d be the physical height of the calibration unit in the calibration board. Taking the dot calibration board image as an example, if we calculate the actual pixel diameter of the calibration unit, then d world This is the physical diameter of the calibration unit in the calibration plate.

[0065] Step S203: Calculate the rotation angle of the calibration board image based on the estimated pixel size.

[0066] In this embodiment, the estimated pixel size of the calibration unit in the calibration board image includes the estimated pixel width and the estimated pixel height. The estimated pixel width and the estimated pixel height may be equal or unequal. The process of calculating the rotation angle of the calibration board image includes:

[0067] Step S203a: Obtain the center point of the calibration plate image.

[0068] Step S203b: Based on the center point of the calibration board image and the estimated pixel size, determine the first region of interest along the first direction and the second region of interest along the second direction.

[0069] Step S203c: In the first region of interest, determine the first edge line of the calibration unit in the calibration plate image along the first direction, and in the second region of interest, determine the second edge line of the calibration unit in the calibration plate image along the second direction.

[0070] In step S203d, the tilt angle of the first edge line is determined as the first rotation angle of the calibration plate image, and the tilt angle of the second edge line is determined as the second rotation angle of the calibration plate image.

[0071] Specifically, firstly, the center point of the calibration board image is obtained based on its size. Then, with the center point of the calibration board image as the center, a first region of interest (ROI) with a width of K1 estimated pixels and a height of K2 estimated pixels is determined. A second region of interest (ROI) with a width of K1 estimated pixels and a height of K2 estimated pixels is also determined, with the center point of the calibration board image as the center. The values ​​of K1 and K2 can be set according to actual conditions; K1 and K2 can be the same or different. Figure 7 shows a second scene schematic diagram of an image distortion correction method according to a second embodiment of this disclosure. As shown in Figure 7, taking a checkerboard calibration board image as an example, if the center point of the calibration board image is point E, and the first direction is horizontal, then with point E as the center, a first region of interest (ROI) with a width of 3 estimated pixels and a height of 1 estimated pixel is determined. Figure 8 shows a schematic diagram of a third scenario of an image distortion correction method according to the second embodiment of this disclosure. As shown in Figure 8, taking a checkerboard calibration board image as an example, if the center point of the calibration board image is point E, and the second direction is a vertical direction perpendicular to the first direction, then a second region of interest (ROI2) with a width of 1 estimated pixel and a height of 3 estimated pixels is determined with point E as the center. It should be emphasized that, for ease of display, diagonal lines are used to replace the black fill in the checkerboard in Figures 7 and 8. In addition, in the dotted checkerboard image, the estimated pixel width and estimated pixel height can be the estimated pixel diameter of a dot or the estimated pixel distance between the centers of two adjacent dots, etc. The method of determining the region of interest in the dotted calibration board image is the same as that in the checkerboard calibration board image, and will not be repeated here.

[0072] Specifically, after determining the first region of interest and the second region of interest, within the first region of interest, a first edge line of the calibration unit along the first direction in the calibration board image is determined, and within the second region of interest, a second edge line of the calibration unit along the second direction in the calibration board image is determined. The tilt angle of the first edge line is determined as the first rotation angle of the calibration board image, and the tilt angle of the second edge line is determined as the second rotation angle of the calibration board image. As shown in Figure 7, since the first region of interest (ROI1) is along the horizontal direction (first direction), the first ROI1 is scanned along the vertical direction (second direction) to obtain edge points. Edge points are pixels along the second direction whose average grayscale value between their adjacent first and second pixel sets is greater than a first preset threshold. Specifically, if the difference between the average grayscale value of the n pixels above a pixel and the average grayscale value of the n pixels below it is greater than the first preset threshold, then that pixel is identified as an edge point. Linear fitting is then performed on the edge points to obtain the first edge line FG. The tilt angle of the first edge line FG is the first rotation angle of the calibration plate image relative to the first direction. As shown in Figure 8, in the second region of interest (ROI2), the second edge line HI of the calibration unit along the second direction is determined. The determination method for the second edge line HI is similar to that of the first edge line FG. The tilt angle of the second edge line HI is the second rotation angle of the calibration plate image relative to the second direction. It should be emphasized that in the dot calibration plate image, the first edge line in the first region of interest can be the line connecting the centers of the two farthest dots in the first region of interest; the second edge line in the second region of interest can be the line connecting the centers of the two farthest dots in the second region of interest.

[0073] In the second embodiment of this disclosure, the actual pixel size of the calibration unit in the calibration board image is calculated based on the physical size of the calibration unit in the calibration board, and the rotation angle of the calibration board image is calculated based on the estimated pixel size of the calibration unit in the calibration board image. The accurate actual pixel size and rotation angle can ensure that a more accurate virtual standard grid is subsequently established, thereby further improving the accuracy of the image distortion correction result.

[0074] Figure 9 shows a flowchart of an image distortion correction method according to a third embodiment of the present disclosure. As shown in Figure 9, step S102 extracts corner points from the calibration plate image to obtain target corner points, including:

[0075] Step S301: Perform corner detection on the calibration board image to obtain the detection results.

[0076] In this embodiment, different corner detection methods are selected for different calibration board images. For example, for a checkerboard calibration board image, a corner detection algorithm is used to detect the vertices of each grid in the calibration board image to obtain the target corner. The corner detection algorithm may include the Harris corner detection algorithm and the Shi-Tomasi corner detection algorithm, etc. For a circular calibration board image, the findCirclesGrid function in OpenCV is used to detect the coordinates of the center of each circle to obtain the target corner.

[0077] Step S302: Filter the regions in the detection results where the corner density is greater than the second preset threshold to obtain the corner region.

[0078] In this embodiment, since the detection results obtained by the corner detection method may have multiple detection results corresponding to one real corner, it is also necessary to filter the areas in the detection results where the corner density is greater than the second preset threshold to obtain the corner region. It can be understood that the target corner is located in the corner region.

[0079] In one implementation, when filtering corner regions, firstly, the detection results are projected along the direction corresponding to a first rotation angle based on the actual pixel size to obtain a first projection array. Then, the detection results are projected along the direction corresponding to a second rotation angle based on the actual pixel size to obtain a second projection array. Next, based on the grayscale values ​​of the pixels in the first and second projection arrays, regions with a corner density greater than a first preset threshold are filtered to obtain corner regions. Specifically, the detection results are generally displayed in a binarized form. For example, the grayscale value corresponding to a corner detected in the calibration plate image can be 255, and the grayscale value corresponding to a non-corner pixel can be 0. The first projection array array1 obtained by projecting the detection results along the direction corresponding to the first rotation angle can be:

[0080] array1=[0,0,0,0,0,255,255,0,0,0,0,0,255,255,255,0,255,0,0,0,0,0…],

[0081] It is understandable that in the first projection array `array1`, there is a region with dense corner points approximately every other actual pixel size. Regions with a corner point density greater than a first preset threshold can be defined as corner point regions. For example, if the grayscale values ​​corresponding to the 5th and 6th positions in `array1` are both 255, then the regions containing their corresponding two corner points can be considered corner point regions. Similarly, if the grayscale values ​​corresponding to the 12th, 13th, and 14th positions in `array1` are all 255, then the regions containing their corresponding three corner points can be considered corner point regions. It should be emphasized that, for ease of display, `array1` uses a length of 5 pixels for the actual pixel size. In practical applications, the actual pixel size and the number of pixels contained within the corner point regions can be much larger. Furthermore, the method for determining corner point regions in the second projection array is similar to that in the first projection array, and will not be elaborated upon here.

[0082] Step S303: Determine the target corner point based on the corner point region and the actual pixel size.

[0083] In this embodiment, taking any corner point in the corner region as the center, a third region of interest (ROI) along the first direction and a fourth region of interest (ROI) along the second direction are determined based on the actual pixel size. Within the third ROI, a third edge line of the calibration unit in the calibration board image along the first direction is determined, and within the fourth ROI, a fourth edge line of the calibration unit in the calibration board image along the second direction is determined. Finally, the intersection of the third and fourth edge lines is determined as the target corner point. The method for determining the third ROI is similar to that of the first ROI, the method for determining the fourth ROI is similar to that of the second ROI, the method for determining the third edge line is similar to that of the first edge line, and the method for determining the fourth edge line is similar to that of the second edge line; these methods will not be repeated here.

[0084] Figure 10 shows a scene diagram of an image distortion correction method according to the third embodiment of the present disclosure. As shown in Figure 10, if a third region of interest (ROI) 3 and a fourth region of interest (ROI) 4 are determined with any corner point in the corner region as the center, and a third edge line JK is determined in the third region of interest ROI 3 and a fourth edge line LM is determined in the fourth region of interest ROI 4, then the intersection point N of the third edge line JK and the fourth edge line LM is a target corner point.

[0085] In the third embodiment of this disclosure, the corner detection results corresponding to the calibration plate image are projected, and the corner area with a high corner density is determined based on the projection results. The target corner is then determined within the corner area, which can yield more accurate target corners and further improve the accuracy of the image distortion correction results.

[0086] In the fourth embodiment of this disclosure, step S103, which establishes a virtual standard grid corresponding to the calibration board image based on parameter information and target corner points, includes:

[0087] A specified corner point is determined among the target corner points. The specified corner point is the target corner point with the smallest distance from the center point of the calibration board image. Based on the specified corner point, rotation angle and actual pixel size, a virtual standard grid corresponding to the calibration board image is established. That is, with the specified corner point as the center, the rotation angle as the rotation angle of the virtual standard grid, and the actual pixel size as the pixel size of each sub-grid in the virtual standard grid, a standard and distortion-free virtual standard grid is established.

[0088] In this embodiment, the distances between all target corner points and the center point of the calibration board image are first calculated, and the target corner point with the smallest distance is determined as the designated corner point. Then, with the designated corner point as the center, a vertex is determined at intervals of actual pixel width in the third direction corresponding to the rotation angle, until the vertex covers all target corner points in the third direction. Then, with the determined vertex as the center, a vertex is determined at intervals of actual pixel height in the vertical direction of the third direction, until the vertex covers all target corner points in the vertical direction of the third direction. Finally, the vertices are connected along the third direction and the vertical direction of the third direction to obtain the virtual standard grid corresponding to the calibration board image. Specifically, the rotation angle can be a first rotation angle or a second rotation angle. For example, if the first rotation angle is larger, the first rotation angle can be used here. In the checkerboard calibration board, the actual pixel width and actual pixel height of each calibration unit can be equal or unequal. In the dot calibration board image, the actual pixel width and actual pixel height of each calibration unit can be the actual pixel diameter of a dot or the actual pixel distance between the centers of two adjacent dots, etc.

[0089] Figure 11 shows a scenario diagram of an image distortion correction method according to the fourth embodiment of this disclosure. As shown in Figure 11, if point O is a specified corner point and the rotation angle is a first rotation angle, then with point O as the center, in the third direction corresponding to the first rotation angle, a vertex is determined at intervals of actual pixel width until the vertex covers all target corner points in the third direction. That is, all vertices in the dashed box in Figure 11 are first determined, and then with the vertex in the dashed box as the center, in the vertical direction of the third direction, a vertex is determined at intervals of actual pixel height until the vertex covers all target corner points in the vertical direction of the third direction. Finally, all vertices in Figure 11 are obtained. These vertices are the vertices of the virtual standard grid. Connecting these vertices will yield the virtual standard grid.

[0090] In the fourth embodiment of this disclosure, a virtual standard grid corresponding to the calibration board image is established based on the precise target corner points and the actual pixel size of the calibration unit in the calibration board image. Subsequently, the corner point offset vector map corresponding to the calibration board image can be obtained based on the virtual standard grid, and the distortion table can be determined based on the corner point offset vector map, thereby calibrating the distortion table using a non-parametric calibration method, which is suitable for image acquisition devices with varying external parameters.

[0091] In the fifth embodiment of this disclosure, step S105 involves performing inverse linear interpolation on a preset table corresponding to the calibration board image based on multiple corner offset vector maps to obtain a distortion table, including:

[0092] Multiple corner offset vector maps are superimposed to obtain an overall offset vector map; based on the overall offset vector map, a reverse linear interpolation formula for a preset table is constructed; the reverse linear interpolation formula is solved using the weighted least squares method to obtain the distorted table.

[0093] In this embodiment, multiple corner offset vector maps corresponding to multiple calibration board images are first superimposed to obtain an overall offset vector map. Figure 12 shows a schematic diagram of a first scenario of an image distortion correction method according to the fifth embodiment of this disclosure. As shown in Figure 12, the solid line table and the dashed line table each correspond to a corner offset vector map. Each target corner point is marked with its coordinates and a first offset. By superimposing the solid line table and the dashed line table, the overall offset vector map shown in Figure 12 can be obtained. In practical applications, the number of corner offset vector maps can be greater.

[0094] In this embodiment, it is also necessary to construct a reverse linear interpolation formula for the preset table based on the overall offset vector map. Figure 13 shows a second scene schematic diagram of an image distortion correction method according to the fifth embodiment of this disclosure. As shown in Figure 13, the lines in the overall offset vector map are all represented by dashed lines, and the preset table corresponding to the calibration plate image is represented by solid lines. The calculation is based on the number of vertices in the horizontal and vertical directions. The preset table in Figure 13 is a 3×3 table. Q 11 Q 21 Q 12 and Q 22 Let Q be the four vertices of a sub-table f in a preset table, and let P1 be a target corner point falling into sub-table f. It can be understood that sub-table f has multiple target corner points, and the first offset of each target corner point is known. Therefore, based on the first offsets of the multiple target corner points in sub-table f, a reverse linear interpolation formula can be constructed for the preset table, and the four vertices Q of sub-table f can be calculated using reverse linear interpolation. 11 Q 21 Q 12 and Q 22 The second offset.

[0095] In one possible implementation, the inverse linear interpolation formula is as follows: Formula (I):

[0096] wAX = wB, Formula (1)

[0097] Where w is the weight, A is the linear interpolation matrix constructed based on the target corner points in the overall offset vector map, X is the second offset corresponding to the vertex of the preset table, i.e., the unknown quantity to be solved, and B is the matrix corresponding to the target corner points in the overall offset vector map. If the first offset of point P1 in Figure 13 is P1 (x1,y1) Let x1 be the first offset of point P1 on the horizontal axis and y1 be the first offset of point P2 on the vertical axis. Then, the reverse linear interpolation formula corresponding to the preset table in Figure 12 is as follows: Formula (II):

[0098]

[0099] That is, let the formula (1) It is important to emphasize that, due to the constraints between meshes, vertex Q 11 The second offset is only related to the target corner point within sub-table f, while vertex Q 22 The second offset is related to all target corner points within sub-tables f, g, h, and i. Therefore, in formula (ii), Q is not only obtained from the target corner points within sub-table f. 11 Q 21 Q 12 and Q 22 The second offset is not calculated by the matrix, but by calculating the offset of all vertices in the preset table in Figure 13 at once from all target corner points within sub-tables f, g, h, and i. It is a 9x1 matrix, and correspondingly, the matrix This is an n x 9 matrix, where n is the number of target corner points in the overall offset vector map; the matrix This results in an n-row, 1-column matrix. After constructing the inverse linear interpolation formula as shown in Formula (II), the distortion table can be obtained by solving Formula (II) using the weighted least squares method.

[0100] Figure 14 shows a flowchart of an image distortion correction method according to a sixth embodiment of this disclosure. As shown in Figure 14, the inverse linear interpolation formula is solved using the weighted least squares method to obtain a distortion table, including:

[0101] The inverse linear interpolation formula is solved using the weighted least squares method to obtain the solution result. The total error corresponding to the solution result is then judged to determine if it meets a preset condition, resulting in a first judgment result. If the first judgment result is yes, a distortion table is determined based on the solution result, that is, the second offset corresponding to each vertex in the solution result is marked in the preset table, thus obtaining the distortion table. If the first judgment result is no, the weights in the inverse linear interpolation formula are updated based on the total error, and the inverse linear interpolation formula is solved based on the updated weights until the first judgment result is yes. Specifically, the weights in the inverse linear interpolation formula can be updated according to the following formula (III):

[0102]

[0103] Where err is the total error and σ is the error parameter.

[0104] In one possible implementation, the method for determining whether the sum of errors corresponding to the solution result meets a preset condition is as follows: if the solution result is the result of the first iteration, then it is determined whether the sum of errors corresponding to the solution result is less than a third preset threshold, thus obtaining a first determination result; if the solution result is not the result of the first iteration, then it is determined whether the sum of errors corresponding to the solution result is less than a fourth preset threshold, thus obtaining a first determination result. The third and fourth preset thresholds can be set according to actual conditions. Preferably, the third preset threshold can be 1e. -7 The fourth preset threshold can be 1e -5 .

[0105] In the fifth and sixth embodiments of this disclosure, an overall offset vector map is obtained by superimposing multiple corner offset vector maps, and an inverse linear interpolation formula is constructed based on the overall offset vector map. Then, the inverse linear interpolation formula is solved by weighted least squares method to obtain a more accurate distortion table, which further ensures the accuracy of the image distortion correction result.

[0106] In the seventh embodiment of this disclosure, step S106, based on the distortion table, performs distortion correction on the image to be corrected to obtain the image distortion correction result, including:

[0107] Based on the distortion table, a forward linear interpolation formula is constructed for the image to be corrected; the forward linear interpolation formula is solved to obtain the third offset of the pixels in the image to be corrected; based on the third offset, distortion correction is performed on the image to be corrected to obtain the image distortion correction result.

[0108] In this embodiment, the distortion table characterizes the distortion trend of the image under the current image acquisition device. The image to be corrected is the image captured by the current image acquisition device that needs distortion correction. Therefore, based on the distortion table, a forward linear interpolation formula for the image to be corrected can be constructed. Then, the forward linear interpolation formula is solved to obtain the third offset of the pixels in the image to be corrected. Distortion correction is then performed on the image to be corrected based on the third offset to obtain the image distortion correction result. It should be emphasized that the calibration board image and the image to be corrected are captured by the current image acquisition device under the same conditions; therefore, the distortion table and the image to be corrected are the same size.

[0109] Figure 15 shows a scenario diagram of an image distortion correction method according to the seventh embodiment of this disclosure. As shown in Figure 15, if the distortion table is a 3×3 table as shown in Figure 13, then the distortion table is superimposed on the image to be corrected. The four vertices Q of the sub-table f of the distortion table can be used as a reference. 11 Q 21 Q 12 and Q 22 The second offset is calculated by using positive linear interpolation to calculate the third offset of each pixel in the image to be corrected in sub-table f. The third offset of each pixel in the image to be corrected in all sub-tables is calculated in the same way. The image distortion correction result can be obtained by performing an affine transformation on the image to be corrected based on the third offset.

[0110] In one possible implementation, the forward linear interpolation formula is as follows:

[0111] P = (1-x)(1-y)Q 11 +(1-x)yQ 21 +x(1-y)Q 12 +yQ 22 ,

[0112] Where P is the third offset of a pixel in the image to be corrected, x is the x-coordinate of point P, y is the y-coordinate of point P, and Q... 11 Q 21 Q 12 and Q 22 These are the second offsets of the four vertices of the sub-table containing point P in the distorted table.

[0113] In the seventh embodiment of this disclosure, the third offset of the pixel in the image to be corrected is first calculated by positive linear interpolation according to the distortion table. Then, the distortion correction of the image to be corrected is performed according to the third offset to obtain the image distortion correction result. This not only simplifies the correction process but also achieves higher accuracy requirements.

[0114] It should be emphasized that the first preset threshold, the second preset threshold, the third preset threshold and the fourth preset threshold in this disclosure can all be set according to the actual situation, and this disclosure does not limit them.

[0115] Figure 16 shows a schematic diagram of an image distortion correction device according to the eighth embodiment of this disclosure. As shown in Figure 16, the device mainly includes:

[0116] The module 10 is used to acquire multiple calibration board images and parameter information of the calibration board images; the corner extraction module 11 is used to extract corner points from the calibration board images to obtain target corner points; the creation module 12 is used to create a virtual standard grid corresponding to the calibration board image based on the parameter information and target corner points; the calculation module 13 is used to calculate the first offset between the target corner point and the vertex of the virtual standard grid corresponding to its position to obtain the corner offset vector map corresponding to the calibration board image; the distortion table generation module 14 is used to perform inverse linear interpolation on the preset table corresponding to the calibration board image based on multiple corner offset vector maps to obtain the distortion table; the distortion correction module 15 is used to perform distortion correction on the image to be corrected based on the distortion table to obtain the image distortion correction result.

[0117] In one embodiment, the acquisition module 10 includes: an acquisition submodule, used to acquire the physical dimensions of the calibration units in the calibration board and the estimated pixel dimensions of the calibration units in the calibration board image; a calculation submodule, used to calculate the actual pixel dimensions of the calibration units in the calibration board image based on the physical dimensions; and a determination submodule, used to determine the rotation angle of the calibration board image based on the estimated pixel dimensions.

[0118] In one possible implementation, the calculation submodule is further configured to: determine the overall physical size of a preset number of calibration units in the calibration board based on the physical size; calculate the overall pixel size of the preset number of calibration units in the calibration board image; calculate the pixel equivalent of the calibration board image based on the overall physical size and the overall pixel size; and calculate the actual pixel size of the calibration units in the calibration board image based on the physical size and the pixel equivalent.

[0119] In one possible implementation, the determining submodule is further configured to: acquire the center point of the calibration board image; determine a first region of interest along a first direction and a second region of interest along a second direction based on the center point of the calibration board image and the estimated pixel size; within the first region of interest, determine a first edge line of the calibration unit in the calibration board image along the first direction, and within the second region of interest, determine a second edge line of the calibration unit in the calibration board image along the second direction; determine the tilt angle of the first edge line as a first rotation angle of the calibration board image, and determine the tilt angle of the second edge line as a second rotation angle of the calibration board image.

[0120] In one possible implementation, the estimated pixel size includes the estimated pixel width and the estimated pixel height. The determining submodule is further configured to: determine a first region of interest with a width of K1 estimated pixels and a height of K2 estimated pixels, centered on the center point of the calibration board image; and determine a second region of interest with a width of K1 estimated pixels and a height of K2 estimated pixels, centered on the center point of the calibration board image.

[0121] In one embodiment, the determining submodule is further configured to: scan the first region of interest along the second direction to obtain edge points, wherein the edge points are pixels whose grayscale average values ​​of the first set of pixels and the second set of pixels adjacent to them along the second direction are greater than a first preset threshold; and perform linear fitting on the edge points to obtain a first edge line.

[0122] In one embodiment, the corner extraction module includes: a detection submodule for detecting corners in the calibration board image and obtaining detection results; a filtering submodule for filtering regions in the detection results where the corner density is greater than a second preset threshold to obtain corner regions; and a target corner determination submodule for determining target corners based on the corner regions and actual pixel sizes.

[0123] In one embodiment, the filtering submodule is further configured to: project the detection result along the direction corresponding to the first rotation angle according to the actual pixel size to obtain a first projection array; project the detection result along the direction corresponding to the second rotation angle according to the actual pixel size to obtain a second projection array; and filter the regions in the detection result whose corner density is greater than a first preset threshold according to the gray values ​​of the pixels in the first projection array and the second projection array to obtain corner regions.

[0124] In one embodiment, the target corner point determination submodule is further configured to: determine a third region of interest along a first direction and a fourth region of interest along a second direction, centered on any corner point in the corner point region and based on the actual pixel size; within the third region of interest, determine a third edge line of the calibration unit in the calibration board image along the first direction, and within the fourth region of interest, determine a fourth edge line of the calibration unit in the calibration board image along the second direction; and determine the intersection of the third edge line and the fourth edge line as the target corner point.

[0125] In one embodiment, the parameter information includes the rotation angle of the calibration board image and the actual pixel size of the calibration unit in the calibration board image. The establishment module 12 includes: a specified corner point determination submodule, used to determine a specified corner point among the target corner points, wherein the specified corner point is the target corner point with the smallest distance from the center point of the calibration board image; and an establishment submodule, used to establish a virtual standard grid corresponding to the calibration board image based on the specified corner point, the rotation angle and the actual pixel size.

[0126] In one possible implementation, the actual pixel size includes the actual pixel width and the actual pixel height. The establishment submodule is further configured to: determine a vertex at intervals of actual pixel width in the third direction corresponding to the rotation angle, centered on a specified corner point, until the vertex covers all target corner points in the third direction; determine a vertex at intervals of actual pixel height in the vertical direction of the third direction, centered on the determined vertex, until the vertex covers all target corner points in the vertical direction of the third direction; and connect the vertices along the third direction and the vertical direction of the third direction to obtain a virtual standard grid corresponding to the calibration board image.

[0127] In one embodiment, the distortion table generation module 14 includes: an overlay submodule for overlaying multiple corner offset vector maps to obtain an overall offset vector map; a first construction submodule for constructing a reverse linear interpolation formula for a preset table based on the overall offset vector map; and a first solution submodule for solving the reverse linear interpolation formula using the weighted least squares method to obtain the distortion table. The reverse linear interpolation formula is as follows: wAX = wB, where w is the weight, A is the linear interpolation matrix constructed based on the target corner points in the overall offset vector map, X is the second offset corresponding to the vertices of the preset table, and B is the matrix corresponding to the target corner points in the overall offset vector map.

[0128] In one possible implementation, the first solving submodule is further configured to: determine whether the total error corresponding to the solving result meets a preset condition, and obtain a first judgment result; if the first judgment result is yes, then determine the distortion table based on the solving result; if the first judgment result is no, then update the weights in the inverse linear interpolation formula based on the total error, and solve the inverse linear interpolation formula based on the updated weights, until the first judgment result is yes.

[0129] In one possible implementation, the first solving submodule is further configured to: if the solving result is the result of the first iteration, determine whether the total error corresponding to the solving result is less than a third preset threshold, and obtain a first judgment result; if the solving result is not the result of the first iteration, determine whether the total error corresponding to the solving result is less than a fourth preset threshold, and obtain a first judgment result.

[0130] In one embodiment, the distortion correction module 15 includes: a second construction submodule, used to construct a forward linear interpolation formula for the image to be corrected based on a distortion table; a second solving submodule, used to solve the forward linear interpolation formula to obtain a third offset of pixels in the image to be corrected; and a distortion correction submodule, used to perform distortion correction on the image to be corrected based on the third offset to obtain an image distortion correction result, wherein the forward linear interpolation formula is as follows: P = (1-x)(1-y)Q 11 +(1-x)yQ 21 +x(1-y)Q12 +xyQ 22 Where P is the third offset of a pixel in the image to be corrected, x is the x-coordinate of point P, y is the y-coordinate of point P, and Q... 11 Q 21 Q 12 and Q 22 These are the second offsets of the four vertices of the sub-table containing point P in the distorted table.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0132] Figure 17 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] As shown in Figure 17, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 can also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0134] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as an image distortion correction method. For example, in some embodiments, an image distortion correction method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of an image distortion correction method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform an image distortion correction method by any other suitable means (e.g., by means of firmware).

[0136] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0141] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0142] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0143] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0144] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image distortion correction method, characterized in that, The method includes: acquiring multiple calibration board images and parameter information of the calibration board images; extracting corner points from the calibration board images to obtain target corner points; establishing a virtual standard grid corresponding to the calibration board images based on the parameter information and the target corner points; calculating a first offset between the target corner point and the vertex of the virtual standard grid corresponding to its position to obtain a corner point offset vector map corresponding to the calibration board images; performing inverse linear interpolation on a preset table corresponding to the calibration board images based on the multiple corner point offset vector maps to obtain a distortion table; and performing distortion correction on the image to be corrected based on the distortion table to obtain an image distortion correction result. The parameter information of the calibration board images is acquired as follows: acquiring the physical size of the calibration unit in the calibration board and the estimated pixel size of the calibration unit in the calibration board image; calculating the calibration board image based on the physical size. The actual pixel size of the calibration unit; the rotation angle of the calibration board image is determined based on the estimated pixel size; wherein, determining the rotation angle of the calibration board image based on the estimated pixel size includes: obtaining the center point of the calibration board image; determining a first region of interest along a first direction and a second region of interest along a second direction based on the center point of the calibration board image and the estimated pixel size; within the first region of interest, determining a first edge line of the calibration unit in the calibration board image along the first direction, and within the second region of interest, determining a second edge line of the calibration unit in the calibration board image along the second direction; determining the tilt angle of the first edge line as the first rotation angle of the calibration board image, and determining the tilt angle of the second edge line as the second rotation angle of the calibration board image.

2. The method according to claim 1, characterized in that, The step of calculating the actual pixel size of the calibration unit in the calibration board image based on the physical size includes: determining the overall physical size of a preset number of calibration units in the calibration board based on the physical size; calculating the overall pixel size of the preset number of calibration units in the calibration board image; calculating the pixel equivalent of the calibration board image based on the overall physical size and the overall pixel size; and calculating the actual pixel size of the calibration unit in the calibration board image based on the physical size and the pixel equivalent.

3. The method according to claim 1, characterized in that, The estimated pixel size includes the estimated pixel width and the estimated pixel height. Determining the first region of interest along the first direction and the second region of interest along the second direction includes: centering on the center point of the calibration board image, determining... A first region of interest is defined with an estimated pixel width as its width and an estimated pixel height as its height; a region of interest is defined with the center point of the calibration board image as its center, having an estimated pixel width as its width and an estimated pixel height as its height. A second region of interest with an estimated pixel height of [high].

4. The method according to claim 1, characterized in that, Determining the first edge line of the calibration unit along the first direction in the calibration plate image includes: scanning the first region of interest along the second direction to obtain edge points, wherein the edge points are pixels along the second direction whose grayscale average value difference between the adjacent first pixel set and the second pixel set is greater than a first preset threshold; and performing linear fitting on the edge points to obtain the first edge line.

5. The method according to claim 1, characterized in that, The step of extracting corner points from the calibration board image to obtain target corner points includes: performing corner point detection on the calibration board image to obtain detection results; filtering out regions in the detection results where the corner point density is greater than a second preset threshold to obtain corner point regions; and determining target corner points based on the corner point regions and the actual pixel size.

6. The method according to claim 5, characterized in that, The step of filtering the detection results to obtain corner regions by selecting regions with corner density greater than a first preset threshold includes: projecting the detection results along a direction corresponding to a first rotation angle based on the actual pixel size to obtain a first projection array; projecting the detection results along a direction corresponding to a second rotation angle based on the actual pixel size to obtain a second projection array; and selecting regions with corner density greater than a first preset threshold based on the grayscale values ​​of pixels in the first and second projection arrays to obtain corner regions.

7. The method according to claim 5, characterized in that, The step of determining the target corner point based on the corner point region and the actual pixel size includes: taking any corner point in the corner point region as the center, determining a third region of interest along a first direction and a fourth region of interest along a second direction based on the actual pixel size; within the third region of interest, determining a third edge line of the calibration unit in the calibration board image along the first direction, and within the fourth region of interest, determining a fourth edge line of the calibration unit in the calibration board image along the second direction; and determining the intersection of the third edge line and the fourth edge line as the target corner point.

8. The method according to claim 1, characterized in that, The parameter information includes the rotation angle of the calibration board image and the actual pixel size of the calibration unit in the calibration board image. The step of establishing a virtual standard grid corresponding to the calibration board image based on the parameter information and the target corner point includes: determining a specified corner point among the target corner points, wherein the specified corner point is the target corner point with the smallest distance from the center point of the calibration board image; and establishing a virtual standard grid corresponding to the calibration board image based on the specified corner point, the rotation angle, and the actual pixel size.

9. The method according to claim 8, characterized in that, The actual pixel size includes the actual pixel width and the actual pixel height. Establishing the virtual standard grid corresponding to the calibration board image includes: using the specified corner point as the center, determining a vertex at intervals of the actual pixel width in the third direction corresponding to the rotation angle, until the vertex covers all target corner points in the third direction; using the determined vertex as the center, determining a vertex at intervals of the actual pixel height in the vertical direction of the third direction, until the vertex covers all target corner points in the vertical direction of the third direction; connecting the vertices along the third direction and the vertical direction of the third direction to obtain the virtual standard grid corresponding to the calibration board image.

10. The method according to claim 1, characterized in that, The step of performing reverse linear interpolation on a preset table corresponding to the calibration board image based on multiple corner offset vector maps to obtain a distortion table includes: superimposing multiple corner offset vector maps to obtain an overall offset vector map; constructing a reverse linear interpolation formula for the preset table based on the overall offset vector map; and solving the reverse linear interpolation formula using the weighted least squares method to obtain the distortion table.

11. The method according to claim 10, characterized in that, The inverse linear interpolation formula is as follows: ,in, As weight, This is a linear interpolation matrix constructed based on the target corner points in the overall offset vector map. This is the second offset corresponding to the vertex of the preset table. This is the matrix corresponding to the target corner points in the overall offset vector map.

12. The method according to claim 10, characterized in that, The step of solving the inverse linear interpolation formula using the weighted least squares method to obtain the distortion table includes: solving the inverse linear interpolation formula using the weighted least squares method to obtain a solution result; determining whether the total error corresponding to the solution result satisfies a preset condition to obtain a first judgment result; if the first judgment result is yes, then determining the distortion table based on the solution result; if the first judgment result is no, then updating the weights in the inverse linear interpolation formula based on the total error, and solving the inverse linear interpolation formula based on the updated weights, until the first judgment result is yes.

13. The method according to claim 12, characterized in that, The step of determining whether the sum of errors corresponding to the solution result meets a preset condition to obtain a first determination result includes: if the solution result is the result of the first iteration, then determining whether the sum of errors corresponding to the solution result is less than a third preset threshold to obtain the first determination result; if the solution result is not the result of the first iteration, then determining whether the sum of errors corresponding to the solution result is less than a fourth preset threshold to obtain the first determination result.

14. The method according to claim 1, characterized in that, The step of performing distortion correction on the image to be corrected according to the distortion table to obtain the image distortion correction result includes: constructing a forward linear interpolation formula for the image to be corrected according to the distortion table; solving the forward linear interpolation formula to obtain the third offset of the pixels in the image to be corrected; and performing distortion correction on the image to be corrected according to the third offset to obtain the image distortion correction result.

15. The method according to claim 14, characterized in that, The positive linear interpolation formula is as follows: Where P is the third offset of a pixel in the image to be corrected. Let P be the x-coordinate. Let P be the ordinate of point P. 、 、 and These are the second offsets of the four vertices of the sub-table containing point P in the distorted table.

16. An image distortion correction device, characterized in that, The device includes: an acquisition module for acquiring multiple calibration board images and parameter information of the calibration board images; a corner extraction module for extracting corners from the calibration board images to obtain target corners; an establishment module for establishing a virtual standard grid corresponding to the calibration board image based on the parameter information and the target corners; a calculation module for calculating a first offset between the target corner and the vertex of the virtual standard grid corresponding to its position to obtain a corner offset vector map corresponding to the calibration board image; a distortion table generation module for performing inverse linear interpolation on a preset table corresponding to the calibration board image based on the multiple corner offset vector maps to obtain a distortion table; and a distortion correction module for performing distortion correction on the image to be corrected based on the distortion table to obtain an image distortion correction result; wherein, the parameter information of the calibration board image is acquired in the following manner: acquiring the physical dimensions of the calibration units in the calibration board and the calibration units in the calibration board image. Estimate pixel size; calculate actual pixel size of calibration unit in calibration board image based on physical size; determine rotation angle of calibration board image based on estimated pixel size; wherein, determining rotation angle of calibration board image based on estimated pixel size includes: obtaining center point of calibration board image; determining first region of interest along first direction and second region of interest along second direction based on center point of calibration board image and estimated pixel size; determining first edge line of calibration unit in calibration board image along first direction within first region of interest and determining second edge line of calibration unit in calibration board image along second direction within second region of interest; determining tilt angle of first edge line as first rotation angle of calibration board image and determining tilt angle of second edge line as second rotation angle of calibration board image.

17. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-15.

18. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-15.

Images