Optical axis detection and calibration method for ultra-wide-angle lens

Abstract

The invention relates to the technical field of ultra-wide-angle lenses, and discloses an ultra-wide-angle lens optical axis detection calibration method, which comprises the following steps: firstly, selecting a standard wide-angle lens and an image, establishing a camera coordinate system and an image coordinate system, obtaining an image a, obtaining an image b by using a to-be-detected wide-angle lens, comparing the image a with the image b, and calibrating the optical axis of the ultra-wide-angle lens. According to a conversion relation between a camera coordinate system and an image coordinate system, calculating an image edge correction coordinate corresponding to each detection edge point, determining an image correction range of the detection target, establishing a distortion model, according to a preset distortion algorithm of the distortion model, calculating to obtain a distortion coordinate of each image point after distortion, and correcting each distortion point on the distortion coordinate to the image correction coordinate position according to the distortion model, and adjusting lens distortion correction setting or other related parameters of the camera according to an image calibration result. According to the invention, the distortion image is calibrated by setting the coordinate system, so that the calibration of the optical axis is assisted, and the efficiency and the precision are higher.

Description

Technical Field

[0001] The present invention relates to the technical field of ultra-wide-angle lenses, and in particular to an optical axis detection and calibration method for an ultra-wide-angle lens. Background Art

[0002] An ultra-wide-angle lens refers to a wide-angle lens with a particularly wide viewing angle (80-110 degrees). On a 35mm camera, it mostly refers to a 15-20mm lens.

[0003] In electronic products equipped with two or more cameras, it is generally necessary to calibrate the optical axes of the multiple lens assemblies. The most commonly used method for ordinary lens calibration is to prepare a calibration object, such as a calibration plate with a circular hole in the middle of a square, to assist in adjusting the angle between the camera and the calibration object. The angle between the camera and the calibration object is manually adjusted until the optimal installation position is reached. During this process, the completion of the calibration can be determined by observing the overlap of the circular hole positions at different focal lengths.

[0004] The current calibration method or device is relatively complex, and manual external calibration may have certain errors, resulting in poor optical axis calibration effect. Therefore, an optical axis detection and calibration method for an ultra-wide-angle lens is proposed to solve the above-mentioned problems. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the shortcomings of the prior art, the present invention provides an optical axis detection and calibration method for an ultra-wide-angle lens, which has the advantages of simple optical axis adjustment operation, high calibration accuracy, and strong versatility and practicality. It solves the problem that the current methods or devices for implementing calibration are relatively complicated, and manual external calibration will have certain errors, thus resulting in poor optical axis calibration effect.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose of convenient operation and better practicality, the present invention provides the following technical solution: an optical axis detection and calibration method of an ultra-wide-angle lens, comprising the following steps:

[0009] Step 1: Select a standard wide-angle lens, set the image at a fixed position, and establish the camera coordinate system and image coordinate system based on the wide-angle lens and the set image;

[0010] Step 2: photograph the image of the specified position through a standard wide-angle lens and obtain image a;

[0011] Step 3: Obtain the position of the wide-angle lens to be detected in the camera coordinate system, and then use the wide-angle lens to be detected to shoot the image in step 1, thereby obtaining image b;

[0012] Step 4: Compare image a with image b, calculate the image edge correction coordinates corresponding to each detected edge point according to the conversion relationship between the camera coordinate system and the image coordinate system, and determine the image correction range of the detection target;

[0013] Step 5: Establish a distortion model, and calculate the distorted coordinates of each image point after distortion according to the preset distortion algorithm of the distortion model.

[0014] Step 6: At this time, the image is taken again through the wide-angle lens to be corrected to obtain the distorted image c and its distortion coordinates, and each distorted point on the distortion coordinates is corrected to the image correction coordinate position according to the distortion model;

[0015] Step 7: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results;

[0016] Step 8: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results;

[0017] Step 9: Use the calibrated wide-angle lens to shoot the image in step 1 multiple times and compare it with image a.

[0018] Preferably, the origin of the camera coordinate system is located at the optical center of the lens, the x-axis and the y-axis are parallel to the two sides of the camera plane, and the z-axis is perpendicular to the lens plane.

[0019] Preferably, the image coordinate system coincides with the imaging plane, the coordinate origin is usually the center of the imaging plane (the intersection of the optical axis and the imaging plane), and the x-axis and the y-axis are respectively parallel to the two sides of the imaging plane.

[0020] Preferably, image a and image b are used to obtain the intrinsic parameters (including focal length, principal point position, distortion coefficient, etc.) and extrinsic parameters (including rotation matrix and translation vector) of the camera.

[0021] Preferably, the conversion relationship between the camera coordinate system and the image coordinate system is based on the perspective projection principle, mapping points in the three-dimensional space onto a two-dimensional image plane.

[0022] Preferably, the detected edge points are transformed in coordinates. If the image has perspective distortion, it may be necessary to correct the coordinates of these edge points through geometric transformation (such as affine transformation, perspective transformation, etc.).

[0023] Preferably, the distortion algorithm mainly includes correction of radial distortion and tangential distortion.

[0024] Preferably, radial distortion: is the situation where the pixel position farther from the optical axis is deformed due to the lens shape. It can be corrected by quadratic and higher-order polynomial functions related to the distance from the center.

[0025] Preferably, tangential distortion: is the phenomenon that the image is bent due to the non-parallelism between the lens and the image plane.

[0026] (III) Beneficial effects

[0027] Compared with the prior art, the present invention provides an optical axis detection and calibration method for an ultra-wide-angle lens, which has the following beneficial effects:

[0028] 1. The optical axis detection and calibration method of the ultra-wide-angle lens calibrates the distorted image by setting a coordinate system, thereby assisting the calibration of the optical axis, which is more efficient and more accurate.

[0029] 2. The optical axis detection and calibration method of the ultra-wide-angle lens makes image calibration more efficient and convenient by setting a distortion model, thereby further improving the efficiency of optical axis calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the flow chart of the optical axis calibration method of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 , an optical axis detection and calibration method for an ultra-wide-angle lens, comprising the following steps:

[0033] Step 1: Select a standard wide-angle lens, set the image at a fixed position, and establish the camera coordinate system and image coordinate system based on the wide-angle lens and the set image;

[0034] In this embodiment, the camera coordinate system has the origin at the optical center of the lens, the x-axis and the y-axis are parallel to the two sides of the camera plane, and the z-axis is perpendicular to the lens plane.

[0035] Image coordinate system: coincides with the imaging plane. The origin of the coordinate system is usually the center of the imaging plane (the intersection of the optical axis and the imaging plane), and the x-axis and y-axis are parallel to the two sides of the imaging plane respectively.

[0036] Step 2: photograph the image of the specified position through a standard wide-angle lens and obtain image a;

[0037] In this embodiment, a reference plate with known pattern and size is used to obtain the camera's intrinsic parameters (including focal length, principal point position, distortion coefficient, etc.) and extrinsic parameters (including rotation matrix and translation vector, which are used to describe the relationship between the camera coordinate system and the world coordinate system). When in use, the camera equipped with a wide-angle lens needs to be fixed and stabilized, and the image is used as a reference standard to facilitate the calibration of the image coordinates.

[0038] Step 3: Obtain the position of the wide-angle lens to be detected in the camera coordinate system, and then use the wide-angle lens to be detected to shoot the image in step 1, thereby obtaining image b;

[0039] In this embodiment, image b uses image a as a reference standard, so as to quickly determine the distortion of the image and facilitate adjustment of the image.

[0040] Step 4: Compare image a with image b, calculate the image edge correction coordinates corresponding to each detected edge point according to the conversion relationship between the camera coordinate system and the image coordinate system, and determine the image correction range of the detection target;

[0041] In this embodiment, the image correction range is mainly aimed at correcting the distortion caused by wide-angle or ultra-wide-angle lenses. Lens distortion is caused by errors in the lens manufacturing process, while perspective distortion is caused by the characteristics of wide-angle lenses. These distortions will cause originally straight objects to appear as curves in the photo, or appear stretched at the four corners of the picture.

[0042] Step 5: Establish a distortion model, and calculate the distorted coordinates of each image point after distortion according to the preset distortion algorithm of the distortion model.

[0043] In this embodiment, the goal of the distortion model is to describe and correct the geometric distortion in the image, and the constraints include the type of distortion, mathematical expression, and limitation of distortion parameters. Through the distortion model, these distortions can be predicted and corrected, thereby improving the accuracy and usability of the image.

[0044] In addition, the constraint condition is the distortion type: distortion usually includes radial distortion (such as barrel distortion and pincushion distortion), tangential distortion, etc. The distortion model needs to be able to cover these different types of distortion.

[0045] Step 6: At this time, the image is taken again through the wide-angle lens to be corrected to obtain the distorted image c and its distortion coordinates, and each distorted point on the distortion coordinates is corrected to the image correction coordinate position according to the distortion model;

[0046] The distortion coordinates of the distorted image c taken again from a different angle are determined, and the distortion coordinates are introduced into the distortion model, so as to verify the correction capability of the distortion model again.

[0047] Step 7: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results;

[0048] In this embodiment, the optical axis angle of the ultra-wide-angle lens is adjusted by deflecting the image angle, so that the adjustment accuracy is higher.

[0049] Step 8: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results;

[0050] Step 9: Use the calibrated wide-angle lens to shoot the image in step 1 multiple times and compare it with image a.

[0051] exist Figure 1 In the image coordinate system, the origin is at the optical center of the lens, the x-axis and y-axis are parallel to the two sides of the image plane, the z-axis is perpendicular to the lens plane, the image coordinate system coincides with the imaging plane, and the origin is usually the center of the imaging plane (the intersection of the optical axis and the imaging plane), and the x-axis and y-axis are parallel to the two sides of the imaging plane respectively.

[0052] In this embodiment, the camera is calibrated through various coordinate systems, thereby ensuring the optical axis standard of the reference image.

[0053] exist Figure 1 In the image a and image b, the camera's internal parameters (including focal length, principal point position, distortion coefficient, etc.) and external parameters (including rotation matrix and translation vector) are obtained.

[0054] exist Figure 1 In the figure, the transformation relationship between the camera coordinate system and the image coordinate system is based on the perspective projection principle, which maps points in the three-dimensional space to the two-dimensional image plane.

[0055] In this embodiment, specifically, the camera coordinate system takes the optical center of the camera as its origin, the X-axis and Y-axis are respectively parallel to the X-axis and Y-axis of the image coordinate system, the optical axis of the camera is the Z-axis, and the image coordinate system takes the center of the image plane (or the upper left corner, depending on the definition) as its origin, which is used to represent the position of pixels in the image.

[0056] exist Figure 1 In the process, the detected edge points are transformed in coordinates. If the image has perspective distortion, it may be necessary to correct the coordinates of these edge points through geometric transformation (such as affine transformation, perspective transformation, etc.).

[0057] exist Figure 1 In the image processing, the distortion algorithm mainly includes the correction of radial distortion and tangential distortion.

[0058] In this embodiment, radial distortion is the phenomenon that the pixel position farther from the optical axis is deformed due to the shape of the lens, which can be corrected by quadratic and higher-order polynomial functions related to the distance from the center. Tangential distortion is the phenomenon that the image is bent due to the non-parallelism between the lens and the image plane.

[0059] The calculation formula for radial distortion is: r_und i storted = r(1+k_1r^2+k_2r^4+k_3r^6).

[0060] Among them, r_und istorted represents the corrected radial distance, r represents the original radial distance, and k_1, k_2 and k_3 are radial distortion coefficients.

[0061] The calculation formula for tangential distortion is: x_und istorted=x+(2p_1y+p_2(r^2+2x^2)), y_und istorted=y+(p_1(r^2+2y^2)+2p_2x).

[0062] Among them, x_undi storted and y_undi storted represent the corrected x and y coordinates respectively, x and y represent the original coordinates, and p_1 and p_2 are the tangential distortion coefficients.

[0063] Specifically, the calibration of the optical axis may also include physical adjustment of the lens.

[0064] Check the lens installation: Make sure the wide-angle lens is properly installed on the camera body and the interface between the lens and the camera is not loose or damaged.

[0065] Adjust lens tilt and offset (if supported by the lens): For some tilt-shift lenses that support tilt and offset adjustment, special tools can be used to adjust the tilt and offset angles of the lens to further correct the optical axis. This usually needs to be done under the guidance of a professional to avoid damaging the lens or camera.

[0066] The above operation can effectively avoid tangential distortion of the image, thereby ensuring a more accurate optical axis.

[0067] To sum up, the optical axis detection and calibration method of the ultra-wide-angle lens calibrates the distorted image by setting a coordinate system, thereby assisting the calibration of the optical axis, which is more efficient and more accurate. By setting the distortion model, the image calibration is more efficient and convenient, thereby further improving the efficiency of the optical axis calibration.

[0068] Furthermore, when in use, firstly select a standard wide-angle lens, set an image at a fixed position, establish a camera coordinate system and an image coordinate system based on the wide-angle lens and the set image, shoot the image at the specified position through the standard wide-angle lens, and obtain image a, obtain the position of the wide-angle lens to be detected in the camera coordinate system, and then use the wide-angle lens to be detected to shoot the image in step one, thereby obtaining image b, compare image a with image b, calculate the image edge correction coordinates corresponding to each detected edge point according to the conversion relationship between the camera coordinate system and the image coordinate system, determine the image correction range of the detection target, establish a distortion model, and distort the image correction coordinates in step four according to the preset distortion of the distortion model. The algorithm calculates the distortion coordinates of each image point after distortion, and then takes the image again through the wide-angle lens to be corrected to obtain the distorted image c and its distortion coordinates, and corrects each distortion point on the distortion coordinates to the image correction coordinate position according to the distortion model, and adjusts the lens distortion correction settings or other related parameters of the camera according to the image calibration result, and takes the image in step one multiple times through the corrected wide-angle lens and compares it with the image a, which solves the problem that the current method or device for realizing calibration is relatively complicated, and manual external calibration will have certain errors, resulting in poor optical axis calibration effect.

[0069] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0070] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting and calibrating the optical axis of an ultra-wide-angle lens, characterized in that: The steps include: Step 1: Select a standard wide-angle lens, set an image at a fixed position, and establish a camera coordinate system and an image coordinate system based on the wide-angle lens and the set image; Step 2: photographing an image of a designated position through the standard wide-angle lens to obtain image a; Step 3: Obtain the position of the wide-angle lens to be detected in the camera coordinate system, and then use the wide-angle lens to be detected to shoot the image in step 1, so as to obtain image b; Step 4: Compare image a with image b, calculate the image edge correction coordinates corresponding to each detected edge point according to the conversion relationship between the camera coordinate system and the image coordinate system, and determine the image correction range of the detected target; Step 5: Establish a distortion model, and calculate the distorted coordinates of each image point after distortion according to the preset distortion algorithm of the distortion model by using the image correction coordinates in step 4; Step 6: At this time, the image is taken again through the wide-angle lens to be corrected to obtain the distorted image c and its distortion coordinates, and the distortion points on the distortion coordinates are corrected to the image correction coordinate positions according to the distortion model; Step 7: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results; Step 8: Adjust the camera's lens distortion correction settings or other related parameters based on the image calibration results; Step 9: Use the calibrated wide-angle lens to shoot the image in step 1 multiple times and compare it with image a.

2. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The origin of the camera coordinate system is located at the optical center of the lens, the x-axis and the y-axis are parallel to the two sides of the camera plane, and the z-axis is perpendicular to the lens plane.

3. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The image coordinate system coincides with the imaging plane, the coordinate origin is usually the center of the imaging plane (the intersection of the optical axis and the imaging plane), and the x-axis and the y-axis are parallel to the two sides of the imaging plane, respectively.

4. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The image a and the image b are used to obtain the intrinsic parameters (including focal length, principal point position, distortion coefficient, etc.) and extrinsic parameters (including rotation matrix and translation vector) of the camera.

5. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The conversion relationship between the camera coordinate system and the image coordinate system is based on the perspective projection principle, which maps points in the three-dimensional space to the two-dimensional image plane.

6. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The coordinate transformation of the detected edge points may require the correction of the coordinates of these edge points through geometric transformation (such as affine transformation, perspective transformation, etc.) if the image has perspective distortion.

7. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 1, characterized in that: The distortion algorithm mainly includes the correction of radial distortion and tangential distortion.

8. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 7, characterized in that: The radial distortion is caused by the shape of the lens, which causes the pixel position farther from the optical axis to be deformed. It can be corrected by quadratic and higher-order polynomial functions related to the distance from the center.

9. The optical axis detection and calibration method of an ultra-wide-angle lens according to claim 7, characterized in that: The tangential distortion is a phenomenon in which the image is bent due to the non-parallelism between the lens and the image plane.

Images