Lens distortion correction method and device and medium

Abstract

The invention provides a lens distortion correction method and device and a medium, and the method comprises the steps: obtaining a lens parameter, and generating a distortion coefficient curve related to the corresponding relation between a distortion coefficient and a distance according to the lens parameter; according to the distortion coefficient curve, generating a lookup table comprising a mapping relation between the distortion coefficient and a distance, wherein the distance is the distance from a pixel point of the image to a distortion center point; obtaining a coordinate value of a target pixel point in the target image, and calculating a target distance from the target pixel point to a distortion center point of the target image; searching from the lookup table according to a target distance to obtain a target distortion coefficient corresponding to the target distance; and correcting the coordinate value of a target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction. The method is used for finding out the distortion coefficient in combination with the lookup table so as to deduce the distorted coordinate, the calculation amount can be reduced, and the calculation speed can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of semiconductor manufacturing, and particularly to a method, device and medium for lens distortion correction. Background Art

[0002] Lens distortion refers to the deformation of physical imaging caused by the magnification difference of the lens system, manufacturing precision and assembly process deviation, which is mainly divided into radial distortion and tangential distortion. Generally speaking, lens distortion is inevitable. In general, there are two solutions. One is to improve lens distortion by optimizing the lens group structure and selecting good lenses. The other is to correct the distortion through a lens distortion correction algorithm. Generally, the lens distortion correction algorithm refers to radial distortion, which means that the pixel points of the image will shift along the radius direction with the distortion center point as the center of the circle. The mathematical model of distortion can be described by the Taylor series expansion around the principal point. Usually, only the first two terms need to be considered, as follows:

[0003] x0 = x(1 + k1r 2 + k2r 4 )

[0004] y0 = y(1 + k1r 2 + k2r 4 )

[0005] Wherein, x and y are the positions of the original pixel points, x0 and y0 are the positions of the pixel points after distortion, k1 and k2 are the correction term coefficients, and r is the distance between the pixel point and the distortion center.

[0006] It can be seen that the traditional distortion correction algorithm needs to perform a large number of calculations based on the row and column values of the pixel points to obtain the coordinates after distortion, so as to perform address inverse mapping and obtain the corrected image. When the existing lens distortion correction algorithm is applied to digital circuit implementation, a large number of addition and multiplication operations need to be performed on the pixel coordinates, resulting in a large circuit volume, high calculation time and area cost. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and device for lens distortion correction, which can combine a look-up table to find out the distortion coefficient and then deduce its coordinates after distortion, so as to reduce the calculation amount and improve the operation speed.

[0008] To achieve the above object, the present invention provides a method for lens distortion correction, including: obtaining lens parameters, generating a distortion coefficient curve regarding the corresponding relationship between the distortion coefficient and the distance according to the lens parameters; generating a look-up table including the mapping relationship between the distortion coefficient and the distance according to the distortion coefficient curve, where the distance is the distance between the pixel point of the image and the distortion center point; obtaining the coordinate value of the target pixel point in the target image, calculating the target distance from the target pixel point to the distortion center point of the target image; looking up the target distortion coefficient corresponding to the target distance from the look-up table according to the target distance; and correcting the coordinate value of the target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction.

[0009] The beneficial effect of the lens distortion correction method provided by the present invention lies in: generating a distortion coefficient curve regarding the corresponding relationship between the distortion coefficient and the distance according to the lens parameters; generating a look-up table including the mapping relationship between the distortion coefficient and the distance according to the distortion coefficient curve, and combining the look-up table to find the distortion coefficient to perform coordinate correction on the target image, and then deriving its distorted coordinates, which can reduce the calculation amount and improve the operation speed.

[0010] In a possible implementation manner, after obtaining the coordinate value of the target pixel point after distortion correction, it further includes: reconstructing the image pixel points distorted by the lens according to address remapping to obtain the image pixel points after distortion correction.

[0011] In a possible implementation manner, generating a look-up table including the mapping relationship between the distortion coefficient and the distance according to the distortion coefficient curve includes: evenly dividing the distortion coefficient curve corresponding to the effective region into N segments according to the accuracy requirement, recording the distortion coefficient corresponding to the midpoint of each segment of the curve, thereby establishing a look-up table including N distortion coefficients, and the index values of the N distortion coefficients are the serial numbers of the corresponding segments of the curve.

[0012] In a possible implementation manner, the distortion coefficient curve is a function curve reflecting the relationship between the distortion coefficient and the square value of the distance, and satisfies the following formula: LUT = 1 + k1r 2 + k2r 4 , where LUT represents the distortion coefficient, r 2 represents the square value of the distance, and k1 and k2 represent two distortion parameters.

[0013] In another possible implementation manner, looking up the target distortion coefficient corresponding to the target distance from the look-up table includes: calculating the square value of the target distance, shifting the square value of the target distance to the right by a fixed number of bits so that the shifted square value of the target distance is mapped within the standard range to obtain the target index value; and determining the target distortion coefficient corresponding to the target index value from the look-up table.

[0014] Second aspect, the present invention provides a lens distortion correction device, which includes a module / unit for performing the method of any possible design in the first aspect above. These modules / units can be implemented by hardware or by hardware executing corresponding software.

[0015] Third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory. Among them, the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are executed by the processor, the electronic device can implement the method of any possible design in the first aspect above.

[0016] Fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the electronic device is caused to execute the method of any possible design in the first aspect above.

[0017] Fifth aspect, an embodiment of the present application further provides a computer program product. When the computer program product runs on an electronic device, the electronic device is caused to execute the method of any possible design in the first aspect above.

[0018] For the beneficial effects of the above second aspect to fifth aspect, reference can be made to the description in the first aspect above. Description of the Drawings

[0019] Figure 1 It is a schematic flowchart of a lens distortion correction method provided by the present invention;

[0020] Figure 2 It is a schematic diagram of a distortion coefficient curve provided by the present invention;

[0021] Figure 3 It is a schematic flowchart of a lens distortion correction provided by the present invention;

[0022] Figure 4 It is a schematic diagram of a generation method of a lookup table provided by the present invention;

[0023] Figure 5 It is a schematic diagram of the structure of a lens distortion correction device provided by the present invention. Detailed Embodiments

[0024] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. This application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances for the embodiments of this application described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] To solve the problems raised in the background art, this application provides a method for lens distortion correction. Figure 1 is a flowchart of the lens distortion correction method provided by the embodiments of this application. This method can be implemented in a digital circuit, such as Figure 1 shown, this method includes the following steps:

[0027] S101, obtain lens parameters, generate a distortion coefficient curve regarding the corresponding relationship between the distortion coefficient and the distance according to the lens parameters; generate a look-up table including the mapping relationship between the distortion coefficient and the distance according to the distortion coefficient curve, where the distance is the distance between the pixel point of the image and the distortion center point.

[0028] S102, obtain the coordinate value of the target pixel point in the target image, and calculate the target distance from the target pixel point to the distortion center point of the target image.

[0029] S103, look up the target distortion coefficient corresponding to the target distance from the look-up table according to the target distance.

[0030] S104, correct the coordinate value of the target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction.

[0031] In a possible embodiment, after S104, the image pixel points after lens distortion can also be reconstructed according to address remapping to obtain the image pixel points after distortion correction.

[0032] The specific generation method of the lookup table can be as follows: First, the obtained lens parameters may include distortion parameters k1, k2, the focal lengths f in the coordinate axis directions x , f y , and the offset c of the distortion center on the image plane x , c y and other parameter information. After that, taking the square of the distance (r 2 ) from the pixel point of the image to the distortion center point of the image as the abscissa and the distortion coefficient as the ordinate, the distortion coefficient 1 + k1r 2 + k2r 4 with respect to the square of the distance value r 2 is obtained. The distortion coefficient curve can be as shown in Figure 2 . The relationship between the distortion coefficient and the square of the distance value satisfies LUT = 1 + k1r 2 + k2r 4 , where LUT represents the distortion coefficient, r 2 represents the square of the distance value, and k1 and k2 represent two distortion parameters. Then, as shown in the figure, the effective region of the abscissa is equally divided into N segments, and the distortion coefficient curve is also correspondingly divided into N segments. The distortion coefficients corresponding to the midpoints of each segment are recorded to obtain N distortion coefficients. The N distortion coefficients are sorted in sequence from 0 to N - 1, thereby establishing a lookup table including the distortion coefficients and the corresponding index values. This lookup table can be stored in the memory for subsequent use. Exemplarily, the lookup table is shown in Table 1.

[0033] Table 1

[0034] Distortion coefficient LUT(i) Index value (binary) LUT(0) 0 LUT(1) 1 LUT(2) 10 LUT(3) 11 LUT(5) 101 … … LUT(869) 11 0110 0101 LUT(1184) 100 1010 0000

[0035] The above method will be further elaborated below in combination with the flowchart shown in Figure 3 . As described in Figure 3 , it includes the following steps:

[0036] S301, obtain lens parameters through camera calibration, such as distortion parameters k1, k2, the focal lengths f in the coordinate axis directions x , f y , and the offset c of the distortion center on the image plane x , c y .

[0037] S302, taking the square of the distance between the pixel point of the image and the distortion center point as the abscissa and the distortion coefficient as the ordinate, obtain the distortion coefficient 1 + k1r 2+k2r 4 For the curve with respect to the square of the distance value r 2 If the horizontal and vertical coordinates are represented by x and y, it is y = 1 + k1x + k2x 2 Such a fixed quadratic curve.

[0038] As Figure 4 Shown in (a) of [], the square of the distance between the pixel point A and the distortion center point O in the image is denoted as r 2 , and then with r 2 As the abscissa and the distortion coefficient 1 + k1r 2 +k2r 4 As the ordinate, an equation is established to obtain the curve shown in (b) of Figure 4 .

[0039] S303. According to the accuracy requirement, the effective region of the abscissa corresponding to the curve is evenly divided into N segments, which are the 0th segment, the 1st segment... the (N - 1)th segment from left to right. Record the distortion coefficients corresponding to the midpoints of each segment, thereby establishing a look-up table (LUT) including N distortion coefficients. The index values of the N distortion coefficients are the serial numbers of the corresponding segments of the curve.

[0040] S304. Obtain the coordinate values [x, y] of the target pixel point in the input target image. First, calculate the square of the distance r from the target pixel point to the target distortion center point 2 = x 2 + y 2 , and then shift this value to the right by a fixed number of bits to map it entirely into the standard interval corresponding to [0, N - 1] to obtain the target index value.

[0041] S305. Determine the target distortion coefficient corresponding to the target index value from the look-up table.

[0042] S306. Calculate the horizontally distorted coordinates [x0, y0] of the target pixel point according to the distortion coefficient, where x0 = x(1 + k1r 2 + k2r 4 ), y0 = y.

[0043] S307. Adjust the distorted coordinates [x0, y0] of the target pixel point to the pixel coordinate system u = f x x0 + c x According to the camera internal parameter k2, and perform reconstruction processing on the image pixel points distorted by lens distortion according to address remapping to obtain the corrected image pixel points.

[0044] The following Table 2 exemplarily shows the corresponding relationships among the coordinates, the square of the distance value, the index value, and the distortion coefficient (LUT(i)) of different pixel points.

[0045] Table 2

[0046]

[0047]

[0048] For the coefficient i in LUT(i) corresponding to the pixel coordinates (x, y), there can be multiple mappings. In Table 2 of this embodiment, the coefficient For example, the pixel coordinates (20, 30) correspond to It should be understood that when calculating the coefficient i in Table 2, dividing by 2 10 is used. The purpose is to discard the rightmost 10 bits in binary, which is relatively simple and easy to implement; and it is also very intuitive in decimal and has a high approximation. In practical applications, calculating the coefficient i can also use dividing by other powers of 2. Specifically, it can be determined according to the number of binary digits to be discarded. This embodiment does not limit this.

[0049] It should be understood that the squared distance value is related to the magnitudes of the pixel x coordinate and the pixel y coordinate, and is independent of the signs of the pixel x coordinate and the pixel y coordinate. In the above table, it is assumed that the step is 1. In fact, Table 2 is just an exemplary scheme. Specifically, the sparsity of the table can also be changed according to actual needs. For example, the step can be 10, then the pixel coordinates can be (0,0), (20,30), (30,40), (40,40), (50,60), etc.; or for another example, the step can be 5, then the pixel coordinates can be (5,5), (10,10), (15,15), etc.

[0050] As can be seen from the above discussion, the lens distortion correction algorithm based on the look-up table proposed by the present invention only needs to calculate the distance from the pixel coordinate point to the distortion center during operation to complete the correction of the horizontal coordinate value, reducing the amount of calculation. In addition, different precisions can be selected based on different application conditions when establishing the look-up table, and the application is flexible. Moreover, since the symmetry characteristics of the distortion coefficients are fully utilized when establishing the look-up table, a large amount of look-up table resources can be saved, comprehensively considering the efficiency and resource utilization, and it is very suitable for implementation in digital circuits.

[0051] In the above embodiment, the look-up table of the distortion coefficient curve reflects the mapping relationship between the distortion coefficient and the squared distance value. It should be understood that the look-up table of the distortion coefficient curve can also reflect the mapping relationship between the distortion coefficient and the distance, or the look-up table of the distortion coefficient curve can also reflect the mapping relationship between the distortion coefficient and the square root of the distance. For this, no further examples will be shown one by one in this article.

[0052] In some embodiments of the present application, such as Figure 5As shown in the figure, the lens distortion correction device for implementing the above method includes: a lookup table storage unit 501 for storing a lookup table including the mapping relationship between distortion coefficients and distances, where the lookup table is determined based on a distortion coefficient curve regarding the corresponding relationship between distortion coefficients and distances generated using lens parameters;

[0053] a calculation unit 502 for obtaining the coordinate value of a target pixel point in a target image and calculating the target distance from the target pixel point to the distortion center point of the target image;

[0054] a lookup unit 503 for looking up, from the lookup table, the target distortion coefficient corresponding to the target distance according to the target distance;

[0055] a correction unit 504 for correcting the coordinate value of the target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction;

[0056] In a possible embodiment, the device further includes a reconstruction processing unit 505 for: after the correction unit obtains the coordinate value of the target pixel point after distortion correction, performing reconstruction processing on the image pixel points after lens distortion according to address remapping to obtain the image pixel points after distortion correction.

[0057] The above Figure 1 All relevant contents of each step involved in the method embodiment shown above can be cited in the function descriptions of the corresponding unit modules and will not be elaborated here.

[0058] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a computer, the method described in the above method embodiment is implemented.

[0059] The embodiment of the present application also provides an electronic device including a processor and a memory. Among them, the memory is used to store one or more computer programs; when one or more computer programs stored in the memory are executed by the processor, the electronic device can implement the method described in the above method embodiment.

[0060] The present invention also provides a computer program product, and when the computer program product is executed by a computer, the method described in the above method embodiment is implemented.

[0061] Although the embodiments of the present invention have been described in detail above, it is obvious to those skilled in the art that various modifications and changes can be made to these embodiments. However, it should be understood that such modifications and changes all fall within the scope and spirit of the present invention described in the claims. Moreover, the present invention described herein can have other embodiments and can be implemented or realized in various ways.

Claims

1. A method for lens distortion correction, characterized in that, Including: Obtain lens parameters and generate a distortion coefficient curve regarding the correspondence between the distortion coefficient and the distance according to the lens parameters; Generate a look-up table including the mapping relationship between the distortion coefficient and the distance based on the distortion coefficient curve, where the distance is the distance between the pixel point of the image and the distortion center point; Obtain the coordinate value of the target pixel point in the target image, and calculate the target distance from the target pixel point to the distortion center point of the target image; Look up the target distortion coefficient corresponding to the target distance from the look-up table according to the target distance; Correct the coordinate value of the target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction.

2. The method according to claim 1, wherein After obtaining the coordinate value of the target pixel point after distortion correction, it further includes: Perform a reconstruction process on the image pixel points distorted by lens distortion according to address remapping to obtain the image pixel points after distortion correction.

3. The method according to claim 1, characterized in that, Generating a look-up table including the mapping relationship between the distortion coefficient and the distance based on the distortion coefficient curve includes: Divide the distortion coefficient curve corresponding to the valid region into N segments on average according to the accuracy requirement, record the distortion coefficients corresponding to the midpoints of each segment of the curve, so as to establish a look-up table including N distortion coefficients, and the index values of the N distortion coefficients are the serial numbers of the corresponding segments of the curve.

4. The method according to any one of claims 1 to 3, characterized in that The distortion coefficient curve is a function curve reflecting the relationship between the distortion coefficient and the square of the distance, and satisfies the following formula: LUT = 1 + k1r 2 + k2r 4 , where LUT represents the distortion coefficient, r 2 represents the square of the distance, and k1 and k2 represent two distortion parameters.

5. The method according to any one of claims 1 to 3, characterized in that Looking up the target distortion coefficient corresponding to the target distance from the look-up table according to the target distance includes: Calculate the square value of the target distance, and shift the square value of the target distance to the right by a fixed number of bits to obtain the target index value, so that the shifted and deformed square value of the target distance is mapped within the standard range; Determine the target distortion coefficient corresponding to the target index value from the look-up table.

6. A lens distortion correction device, characterized in that, Including: A look-up table storage unit for storing a look-up table including the mapping relationship between the distortion coefficient and the distance, where the look-up table is determined based on the distortion coefficient curve regarding the correspondence between the distortion coefficient and the distance generated by using the lens parameters; A calculation unit for obtaining the coordinate value of the target pixel point in the target image and calculating the target distance from the target pixel point to the distortion center point of the target image; A look-up unit for looking up the target distortion coefficient corresponding to the target distance from the look-up table according to the target distance; A correction unit for correcting the coordinate value of the target pixel point in the target image according to the target distortion coefficient to obtain the coordinate value of the target pixel point after distortion correction.

7. The device according to claim 6, characterized in that, The device further includes a reconstruction processing unit for: after the correction unit obtains the coordinate value of the target pixel point after distortion correction, perform a reconstruction process on the image pixel points distorted by lens distortion according to address remapping to obtain the image pixel points after distortion correction.

8. The device according to claim 6, characterized in that, The look-up table includes N distortion coefficients, and the index values of the N distortion coefficients are the serial numbers of the corresponding segments of the distortion coefficient curve.

9. The device according to any one of claims 6 to 8, characterized in that, The look-up unit looks up the target distortion coefficient corresponding to the target distance from the look-up table, specifically for: Calculate the square value of the target distance, and shift the square value of the target distance to the right by a fixed number of bits to obtain the target index value, so that the shifted and deformed square value of the target distance is mapped within the standard range; Determine the target distortion coefficient corresponding to the target index value from the look-up table.

10. A computer-readable storage medium storing a computer program therein, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images