A fast fisheye image correction method based on distortion stretch factor
By introducing distortion stretching factors into fisheye image correction and establishing a correction optimization model, the problem of high complexity of fisheye image correction in the prior art is solved, and fast and efficient fisheye image correction is achieved, which is suitable for detection application requirements.
Patent Information
- Application Number
- CN202210062723.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-19
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2042-01-19
AI Technical Summary
The existing fisheye image correction methods have problems such as large calculation volume and complex process, which are difficult to meet the requirements of detection applications. Especially when the longitude correction model adapts to images of different distortion degrees, it is easy to lead to excessive correction or insufficient.
The fisheye image rapid correction method based on distortion stretching factor is adopted, and the distortion stretching factor is introduced based on the longitude correction model. By establishing a correction optimization model, the influence of radial distortion is eliminated and correction is performed in both transverse and longitudinal directions.
It realizes efficient correction of fisheye images, improves correction speed and quality, simplifies algorithms, is suitable for applications in embedded and mobile devices, and improves correction accuracy.
Smart Images
Figure CN114331826B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of digital image processing, and in particular to a method for fast correction of fisheye images. Background Art
[0002] The shooting angle of the lens used by ordinary cameras is relatively small. In order to reflect the full content information of the scene being photographed, a fisheye lens can be used. The fisheye lens is a lens with a short focal length and a large angle of view (angle of view is between 180° and 270°). By taking advantage of its large angle of view, a fisheye lens can be used to capture fisheye images that contain more information than images taken with ordinary cameras. In addition, the fisheye lens does not need to rotate or scan when the camera is working. It is small in size and easy to use. Therefore, fisheye lenses are widely used for quality inspection in narrow spaces or areas with limited spatial positions.
[0003] The reason why fisheye lenses can obtain a wider viewing angle range than ordinary camera lenses is that their lens models are different. Most lenses are designed according to the pinhole camera model, which is based on the fact that light propagates in a straight line, and the intersection of two straight lines remains the point where the two straight lines intersect after transformation, that is, the geometric properties remain unchanged, so the captured image can be closely linked to the original scene. However, there is a defect in the lens under this model, that is, the light always propagates in a straight line, making it difficult for the lens to capture objects at the edge of the scene. In the case of limited film (CMOS) size, fisheye lenses can capture more scenes at the edge than ordinary camera lenses.
[0004] At present, the correction methods of fisheye images can be divided into two categories according to whether calibration is required: calibration-based fisheye lens distortion correction methods and correction methods based on projection transformation models. Among them, the correction method based on calibration mainly calibrates the internal and external parameters of the fisheye lens by setting calibration blocks, such as chessboards, concentric circles, dot matrix templates, etc., and corrects the fisheye image on the basis of obtaining the internal and external parameters of the lens. However, this method requires tedious experiments and function calculations around multiple parameters, and the parameters to be solved are also closely related to the type of fisheye lens. The correction method based on the projection transformation model establishes a mathematical model based on the imaging principle of perspective projection, and restores the distorted image to a perspective projection image that conforms to human visual habits by finding the mapping relationship between the fisheye image and the target image. However, this method often has a large amount of calculation and a complex process, and it is difficult to meet the requirements of detection applications.
[0005] The fisheye image correction model based on longitude coordinates, that is, the longitude correction model, describes the functional correspondence between the pixel coordinates in the fisheye image and the pixel coordinates in the corrected image. The longitude correction model transforms the horizontal coordinates of the pixels in the fisheye image to the original position and the vertical coordinates remain unchanged during correction, thereby transforming the circular fisheye image effective area into a square. It is a relatively easy-to-calculate correction method based on the projection transformation model. However, for fisheye images with different degrees of distortion, since the radius of the effective area of the fisheye image is a fixed value, it is impossible to make adaptive adjustments based on the distortion of the fisheye image, resulting in serious over-correction or under-correction after correction using the longitude correction model. In addition, the model only corrects the longitudinal distortion, and there is still very serious distortion in the lateral direction. Summary of the invention
[0006] In order to overcome the deficiencies in the prior art, the present invention provides a method for fast correction of fisheye images based on a distortion stretching factor, which can effectively correct the distortion of fisheye images and improve the correction speed and correction quality of fisheye images.
[0007] In order to achieve the above object, the present invention adopts the following technical solutions:
[0008] A method for quickly correcting fisheye images based on distortion stretching factors comprises the following steps:
[0009] Step 1: Extract the effective area of the fisheye image;
[0010] Step 2: A correction optimization model is established based on the longitude correction model, and a distortion stretching factor for eliminating the influence of radial distortion is introduced into the correction optimization model (i.e., a fisheye image correction model based on the distortion stretching factor is established);
[0011] Step 3: Determine the value of the distortion stretch factor in the correction optimization model, and then establish a mapping relationship between the pixel coordinates in the blank correction image and the sub-pixel coordinates of the effective area of the fisheye image;
[0012] Step 4: Generate a corresponding corrected image of the fisheye image according to the established mapping relationship.
[0013] Preferably, step 1 specifically includes the following steps: using a brightness threshold judgment method to determine the boundary of the effective area of the fisheye image, and then calculating the center and radius of the effective area of the fisheye image to obtain the distortion center and distortion radius of the fisheye image.
[0014] Preferably, in step 2, the longitude correction model is expressed as:
[0015]
[0016] Among them, (x0, y0) is the coordinate of the distortion center in the fisheye image pixel coordinate system, (x, y) is the coordinate of the pixel point in the fisheye image pixel coordinate system, (u, v) is the coordinate of the pixel point in the corrected image pixel coordinate system, and R is the distortion radius of the fisheye image.
[0017] Preferably, in step 2, the correction optimization model is expressed as:
[0018]
[0019] Where k* is the distortion stretch factor.
[0020] Preferably, in step 3, the value of the distortion stretch factor is determined by numerically solving an implicit expression of the distortion stretch factor.
[0021] Preferably, in step 3, the value of the distortion stretch factor is determined by selecting a value from a default value interval [1.12, 1.56].
[0022] Preferably, the step 4 specifically includes the following steps: performing grayscale interpolation after completing the coordinate mapping between the pixel coordinates in the blank corrected image and the sub-pixel coordinates in the effective area of the fisheye image.
[0023] Preferably, the grayscale interpolation adopts a reverse interpolation method of backward mapping.
[0024] A fisheye image fast correction system based on distortion stretch factor, comprising a fisheye image extraction module, a blank image establishment module, a coordinate mapping module and a blank image pixel assignment module;
[0025] The fisheye image extraction module is used to read the input fisheye image and extract the effective area of the fisheye image;
[0026] The blank image creation module is used to generate a blank correction image according to the correction optimization model;
[0027] The coordinate mapping module is used to establish a mapping relationship between pixel coordinates in the blank correction image and sub-pixel coordinates in the effective area of the fisheye image after determining the value of the distortion stretch factor in the correction optimization model;
[0028] The blank image pixel assignment module is used to generate a corresponding corrected image of the fisheye image according to the mapping relationship.
[0029] The beneficial effects of the present invention are embodied in:
[0030] The present invention is based on the longitude correction model and introduces a distortion stretching factor to perform fisheye image correction in both the horizontal and vertical directions on the basis of considering the influence of radial distortion. Compared with the traditional fisheye image correction method based on longitude coordinates, the present invention has the characteristics of simple algorithm, strong real-time performance, easy transplantation of embedded devices and mobile devices, and the difference between the corrected image and the real scene is small, and the correction effect is good.
[0031] Furthermore, the present invention has the flexibility to improve accuracy without significantly increasing the complexity of correction. That is, when the parameters can be determined, only one parameter, the distortion stretch factor, needs to be adjusted according to the solution result to improve the correction accuracy; when the parameters are difficult to determine, the default values of the parameters can be used to select values to improve the correction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 The flowchart of the method for fast correction of fisheye images based on distortion stretching factor in an embodiment of the present invention is shown.
[0033] Figure 2 It is a schematic diagram of establishing a longitude correction model in an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The embodiments are only used to explain the present invention, but not to limit the scope of protection of the present invention.
[0035] Example 1
[0036] like Figure 1 As shown, the method for quickly correcting fisheye images based on distortion stretching factors proposed by the present invention is a method for adaptively correcting fisheye images by adding distortion stretching factors to a longitude correction model, and includes the following steps:
[0037] Step 1: Extract the effective area of the fisheye image.
[0038] Calculate the pixel brightness value I(x,y) of the fisheye image pixel:
[0039] I(x,y)=0.59r(x,y)+0.11g(x,y)+0.3b(x,y)
[0040] Where r(x,y), g(x,y), and b(x,y) are the red, green, and blue components of the pixel with coordinates (x,y) in the fisheye image.
[0041] Set a threshold T, which is used to determine the boundary of the effective area of the fisheye image, and the value of T is between 20-50. Read the pixel brightness values of the 1st, 2nd, 3rd, ... columns of pixels on the left side of the fisheye image, until a pixel brightness value of a pixel in the current column scanned (read column by column) is greater than T, and continue to read the pixel brightness values of the next column of pixels. When the pixel brightness values of the pixels in this column adjacent to the pixels whose pixel brightness values are greater than T in the previous column are still greater than T, it is considered that the left boundary of the effective area of the fisheye image has been scanned (the previous column is taken as the left boundary of the effective area). According to this method, the left, right, top, and bottom boundaries of the effective area of the fisheye image can be obtained respectively, and the distances between these boundaries and the pixel at the lower left corner of the fisheye image are calculated in turn, and the results are recorded as left, right, top, and bottom respectively.
[0042] At this point, the distortion radius R and the distortion center coordinates (x0, y0) of the fisheye image can be directly written out to determine the effective area of the circular fisheye image:
[0043]
[0044] Step 2, taking into account the influence of radial distortion, establishes and solves a distortion correction model to describe the relationship between the distortion center coordinates in the fisheye image pixel coordinate system, the effective area pixel coordinates in the fisheye image pixel coordinate system, and the pixel coordinates in the pixel coordinate system of the corrected image.
[0045] Step 2.1 Establish a fisheye image correction model based on longitude coordinates.
[0046] In a longitude map, the equation for pixel coordinates is:
[0047] x 2 +y 2 =R 2
[0048] Where R is the radius of the longitude circle.
[0049] See also Figure 2 In establishing the longitude correction model, it is assumed that any point A(x, y) on the image before correction becomes point B(u, v) on the image after correction. The projections of these two points on the zero latitude line are C(x, y0) and D(u, y0), respectively. It is known that the center of the effective area is O(x0, y0) and the radius is R. According to the Pythagorean theorem, it can be obtained that:
[0050]
[0051] The longitude correction model can be further obtained:
[0052]
[0053] Step 2.2 introduces the distortion stretch factor k*.
[0054] The radial distortion of fisheye images is the position deviation of the pixels in the image along the radial direction with the center of distortion as the center, which causes the deformation of the scene in the image. The radial distortion of the image is the most important distortion caused by the imaging process of the fisheye lens, and it is also the distortion that has the greatest impact on the correction of the imaging result. The formula is:
[0055]
[0056]
[0057] Among them, (u, v) is the pixel coordinate of the corrected image in the pixel coordinate system, (x, y) is the pixel coordinate of the fisheye image in the pixel coordinate system, r is the distance from the distortion point to the center point, r 2 =x 2 +y 2 ; δ x (x,y) and δ y (x,y) are the distortion values of the x-axis and y-axis respectively; k n is the distortion coefficient (n=1,2,3,…; k1 is the low-order term, k2, k3… are the high-order terms).
[0058] The present invention proposes a distortion stretching factor k* based on the consideration of the influence of radial distortion, which means stretching the distance from all points in the image to the center of the image by k* times.
[0059] Step 2.3 establishes a fisheye image correction model based on the distortion stretching factor.
[0060] Based on the above longitude coordinate model, the fisheye image distortion correction model is established by adding the distortion stretching factor k*:
[0061]
[0062] Step 2.4 calculates the distortion stretch factor.
[0063] According to the formula in step 2.2, let OA = k*R and ignore the high-order terms that have little effect on distortion, and we get:
[0064] k * OA=OB(1+k1OB 2 )
[0065] According to the distance formula between two points:
[0066]
[0067] Further arrangement can obtain the implicit expression for k*:
[0068]
[0069] in
[0070] The fisheye image to be corrected (i.e., the effective area of the fisheye image obtained in step 1) is brought in for specific numerical solution; the value of k* is adjusted according to the actual situation to correct the distortion curve (meridian) with distorted boundaries in the image into a straight line, and the final fisheye image distortion correction model and mapping relationship are obtained according to the value of the distortion stretching factor at this time.
[0071] Step 3 provides a blank correction image according to the above-mentioned fisheye image distortion correction model (the blank correction image matches the fisheye image to be corrected, and the blank correction image is slightly larger than the fisheye image to be corrected), establishes a mapping relationship between the pixel coordinates in the blank correction image and the sub-pixel coordinates in the fisheye image to be corrected, thereby completing the coordinate mapping.
[0072] After completing the coordinate mapping in step 4, grayscale interpolation is performed to obtain the corrected image.
[0073] Specifically, the sub-pixel coordinates in the fisheye image to be corrected that are mapped to the blank corrected image pixel points are reversely interpolated to obtain sub-pixel values, and the obtained sub-pixel values are assigned to the corresponding pixel points in the blank corrected image. In this way, it can be ensured that the image obtained after the fisheye image is corrected (i.e., the corrected image) has less edge loss.
[0074] Example 2
[0075] Taking into account the size of the image to be corrected after extracting the effective area, at this time |x-x0| and |y-y0| are generally between 350-550. The special points in the image (the left vertex, right vertex, upper vertex, and lower vertex of the image) are brought into the calculation of the value of k*. After taking the average, the value of k* calculated to be between 1.12 and 1.56 is more in line with actual needs (for example, k*=1.2). The default value range of k* is thus determined. While simplifying the correction process, a higher correction accuracy can still be guaranteed by adjusting k*.
Claims
1. A method for fast correction of fisheye images based on distortion stretching factor, characterized in that: The following steps are involved: 1) Extract the effective area of the fisheye image; 2) A correction optimization model is established based on the longitude correction model, and a distortion stretching factor is introduced into the correction optimization model to eliminate the influence of radial distortion; 3) Determine the value of the distortion stretch factor in the correction optimization model, and then establish a mapping relationship between the pixel coordinates in the blank correction image and the sub-pixel coordinates of the effective area of the fisheye image; 4) generating a corresponding corrected image of the fisheye image according to the established mapping relationship; In step 2, the longitude correction model is expressed as: In step 2, the correction optimization model is expressed as: Where (x0, y0) is the coordinate of the distortion center in the fisheye image pixel coordinate system, (x, y) is the coordinate of the pixel point in the fisheye image pixel coordinate system, (u, v) is the coordinate of the pixel point in the corrected image pixel coordinate system, R is the distortion radius of the fisheye image, and k* is the distortion stretch factor; In step 3, let OA = k*R, and ignore the high-order terms that have little effect on distortion, and we get: k * OA=OB(1+k1OB 2 ) According to the distance formula between two points: Further arrangement can obtain the implicit expression for the distortion stretch factor: in k* is the distortion stretch factor; The value of the distortion stretch factor is determined by numerically solving the implicit expression of the distortion stretch factor.
2. The method for fast correction of fisheye images based on distortion stretching factor according to claim 1, characterized in that: The step 1 specifically includes the following steps: using a brightness threshold judgment method to determine the boundary of the effective area of the fisheye image, then calculating the center and radius of the effective area of the fisheye image, and obtaining the distortion center and distortion radius of the fisheye image.
3. The method for fast correction of fisheye images based on distortion stretching factor according to claim 1, characterized in that: In step 3, the value of the distortion stretch factor is determined by selecting a value from the default value interval [1.12, 1.56].
4. The method for fast correction of fisheye images based on distortion stretching factor according to claim 1, characterized in that: The step 4 specifically includes the following steps: performing grayscale interpolation after completing the coordinate mapping between the pixel coordinates in the blank corrected image and the sub-pixel coordinates in the effective area of the fisheye image.
5. The method for fast correction of fisheye images based on distortion stretching factor according to claim 4, characterized in that: The grayscale interpolation adopts a reverse interpolation method of backward mapping.
6. A fisheye image fast correction system based on distortion stretch factor, characterized in that: It includes a fisheye image extraction module, a blank image creation module, a coordinate mapping module and a blank image pixel assignment module; The fisheye image extraction module is used to read the input fisheye image and extract the effective area of the fisheye image; The blank image establishment module is used to generate a blank correction image according to the correction optimization model, the correction optimization model is established on the basis of the longitude correction model, and introduces a distortion stretching factor for eliminating the influence of radial distortion; The longitude correction model is expressed as: The correction optimization model is expressed as: Where (x0, y0) is the coordinate of the distortion center in the fisheye image pixel coordinate system, (x, y) is the coordinate of the pixel point in the fisheye image pixel coordinate system, (u, v) is the coordinate of the pixel point in the corrected image pixel coordinate system, R is the distortion radius of the fisheye image, and k* is the distortion stretch factor; Let OA = k*R, and ignore the high-order terms that have little effect on distortion, and we get: k * OA=OB(1+k1OB 2 ) According to the distance formula between two points: Further arrangement can obtain the implicit expression for the distortion stretch factor: in k* is the distortion stretch factor; By numerically solving the implicit expression of the distortion stretch factor, the value of the distortion stretch factor is determined; The coordinate mapping module is used to establish a mapping relationship between pixel coordinates in the blank correction image and sub-pixel coordinates in the effective area of the fisheye image after determining the value of the distortion stretch factor in the correction optimization model; The blank image pixel assignment module is used to generate a corresponding corrected image of the fisheye image according to the mapping relationship.