A fish-eye lens calibration method and system combining two-dimensional and three-dimensional information

Abstract

The application discloses a fish-eye lens calibration method and system combining two-dimensional and three-dimensional information and belongs to the technical field of computer vision and photogrammetry. The method comprises the following steps: collecting two-dimensional checkerboard multi-angle video data and estimating initial camera internal parameters; collecting three-dimensional calibration field images, identifying AprilTag labels and obtaining control point 3D-2D information; performing de-distortion processing on the control points by using the initial internal parameters; estimating initial external parameters by using SVD and RQ decomposition methods; constructing a bundle adjustment optimization model; and performing global optimization on the two-dimensional checkerboard and three-dimensional calibration field observation information by using an MEI camera model to obtain final calibration parameters. The application overcomes the defects of the two-dimensional method, such as lack of absolute spatial constraints, and the three-dimensional method, such as complex operation, and realizes high-precision, high-efficiency and fully-automatic fish-eye camera calibration.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a method and system for calibrating fisheye lenses that combine two-dimensional and three-dimensional information. Background Technology

[0002] A fisheye lens is an extreme wide-angle lens with an extremely short focal length (typically between 6-16mm) and a field of view of 180° or more, creating visually striking images. The wide field of view gives fisheye lenses an exceptionally large field of view, capturing almost everything in sight, especially in narrow, confined spaces (such as small rooms, car interiors, or tunnels), allowing them to capture scenes that ordinary wide-angle lenses cannot. Furthermore, due to its extremely short focal length, a fisheye lens possesses a very deep depth of field. Even at wide apertures, objects from closest to furthest distance remain sharp. This makes focusing incredibly simple, virtually eliminating concerns about out-of-focus issues, making it ideal for scenarios requiring rapid capture. Therefore, fisheye lenses are widely used in action cameras, mobile mapping, real estate advertising, and other industries. However, the exaggerated barrel distortion also places higher demands on camera calibration. Current conventional calibration methods use two-dimensional checkerboard calibration or high-precision three-dimensional calibration field calibration. While two-dimensional checkerboard calibration is convenient, it suffers from drawbacks such as sensitivity to distorted models, the need for multi-angle photography, scale uncertainty, and poor edge calibration accuracy. Conversely, high-precision three-dimensional calibration field methods are costly, inflexible, and complex to operate. In summary, existing technologies fail to achieve a good balance between convenience (e.g., two-dimensional checkerboard) and high precision (e.g., three-dimensional calibration fields). Summary of the Invention

[0003] In view of this, the present invention aims to overcome the shortcomings of the above-mentioned problems in the prior art, and proposes a fisheye lens calibration method and system that combines two-dimensional and three-dimensional information, which solves the problems of complex operation and time-consuming and labor-intensive operation of the high-precision three-dimensional calibration field calibration method, and improves the practicality and efficiency of the method.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0005] In a first aspect, the present invention provides a method for calibrating a fisheye lens that combines two-dimensional and three-dimensional information, comprising the following steps:

[0006] Two-dimensional checkerboard calibration stage: acquire multi-angle video images of the two-dimensional checkerboard calibration board, extract and filter video frames, and use a fisheye camera model to perform initial intrinsic parameter calibration on the filtered images;

[0007] 3D calibration field stage: Collect images of multiple target points set up in the 3D calibration field, identify the AprilTag labels, and automatically identify the labels to obtain the 3D-2D correspondence information of the control points;

[0008] Distortion removal: Combining the 2D coordinates of the initial intrinsic control points, distortion is removed through an iterative method to obtain normalized distortion-free coordinates;

[0009] Extrinsic parameter initialization: Using the distortion-free 2D points and known 3D points, the projection matrix P is solved by SVD decomposition, and then the initial extrinsic parameters of the camera in the calibration field are obtained by RQ decomposition.

[0010] Joint optimization: Construct a bundle adjustment model, fuse the observation data of the two-dimensional checkerboard and the three-dimensional calibration field, and perform global optimization of camera intrinsic parameters, distortion parameters and exterior orientation elements;

[0011] Model prediction and adjustment: Based on the predicted projection positions of control points on the image after optimization, adjustments are made in conjunction with image information, and model parameters are re-optimized;

[0012] Output calibration results: Generate the final camera calibration parameter file.

[0013] Furthermore, the distortion correction process specifically includes:

[0014] Convert the distorted image coordinates to normalized coordinates;

[0015] Calculate the distortion radius;

[0016] The true angle of incidence is determined using Newton's iterative method.

[0017] Calculate the normalized coordinates after distortion removal.

[0018] Furthermore, in the external parameter initialization step:

[0019] Solve the 3×4 projection matrix P using SVD decomposition;

[0020] The initial exterior orientation elements are obtained by decomposing the P matrix into intrinsic and extrinsic parameter matrices through RQ decomposition.

[0021] Furthermore, in the joint optimization step:

[0022] Construct the bundle adjustment loss function;

[0023] Ceres was used to optimize the principal point, focal length, distortion parameters, and exterior orientation elements.

[0024] Furthermore, during the three-dimensional calibration stage, the indoor calibration field layout must meet the following conditions:

[0025] The target points are distributed across multiple non-coplanar spatial planes, including walls, floors, and cylindrical surfaces;

[0026] The target point is the AprilTag tag, which supports automatic recognition and number matching.

[0027] Furthermore, in the initial intrinsic parameter calibration of the selected images using the fisheye camera model, the initial intrinsic parameters are calibrated using the OpenCCV fisheye camera calibration module. The fisheye camera model used by OpenCV is the KB model. After calibration, an 8D vector is obtained to represent the intrinsic parameters, including focal length, principal point, and distortion parameters.

[0028] Secondly, the present invention provides a fisheye lens calibration system that combines two-dimensional and three-dimensional information, comprising:

[0029] The two-dimensional checkerboard calibration module is used to acquire multi-angle video images of the two-dimensional checkerboard calibration board, extract video frames and filter them, and use a fisheye camera model to perform initial intrinsic parameter calibration on the filtered images.

[0030] The 3D calibration field module is used to acquire images of multiple target points set up in the 3D calibration field, identify the AprilTag labels, and automatically identify the labels to obtain the 3D-2D correspondence information of the control points.

[0031] The distortion removal module is used to combine the 2D coordinates of the initial internal control points and remove distortion through an iterative method to obtain normalized distortion-free coordinates.

[0032] The extrinsic parameter initialization module is used to solve the projection matrix P by using the distortion-free 2D points and the known 3D points, and then obtain the initial extrinsic parameters of the camera in the calibration field by RQ decomposition.

[0033] The joint optimization module is used to construct a bundle adjustment model, which integrates the observation data of the two-dimensional checkerboard and the three-dimensional calibration field, and performs global optimization of camera intrinsic parameters, distortion parameters and exterior orientation elements.

[0034] The model prediction and adjustment module is used to adjust the model parameters based on the projection positions of the control points on the image predicted by the optimized model, combined with image information.

[0035] The output calibration results module is used to generate the final camera calibration parameter file.

[0036] Thirdly, the present invention provides an electronic device, including a processor and a memory communicatively connected to the processor and used to store executable instructions of the processor, wherein the processor is used to execute the above-described fisheye lens calibration method combining two-dimensional and three-dimensional information.

[0037] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned fisheye lens calibration method combining two-dimensional and three-dimensional information.

[0038] Compared with existing technologies, the fisheye lens calibration method and system that combines two-dimensional and three-dimensional information described in this invention has the following advantages:

[0039] This invention abandons the traditional approach of relying solely on two-dimensional checkerboard patterns or three-dimensional calibration fields, and innovatively integrates the observation information from both. By designing a specific data acquisition process, it simultaneously acquires multi-angle images of the two-dimensional checkerboard pattern and absolute control point data of the three-dimensional calibration field, and constructs a unified optimization model, thus resolving the core contradiction in existing technologies that cannot achieve a balance between "convenience" and "high precision".

[0040] This invention utilizes the large amount of readily available two-dimensional checkerboard data to first stably calculate the initial values ​​of camera parameters (intrinsic, extrinsic, and distortion). Then, it introduces precise absolute spatial constraints provided by a three-dimensional calibration field to accurately correct and globally optimize the initial parameters (especially the distortion parameters and focal length, which are crucial for edge accuracy). This overcomes the problems of scale uncertainty, poor edge calibration accuracy, and model sensitivity caused by the lack of absolute scale and spatial constraints in pure two-dimensional methods, while avoiding the disadvantages of high cost and inflexibility of pure three-dimensional methods. Attached Figure Description

[0041] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0042] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0043] Figure 2 This is a schematic diagram of the indoor calibration field layout of the present invention. Detailed Implementation

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0045] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] Example 1:

[0047] refer to Figure 1 This invention provides a method for calibrating a fisheye lens that combines two-dimensional and three-dimensional information, specifically including the following:

[0048] (1) Two-dimensional chessboard grid marking:

[0049] Chessboard pattern video data acquisition: First, place the camera on a table or tripod and select the recording mode to start recording. The person collecting the data stands in front of the camera holding a chessboard calibration board and slowly moves the calibration board up, down, left, and right within the camera's field of view. Try to ensure that the positions where the calibration board appears in the shot cover the entire frame. Recording should last approximately one minute, then stop.

[0050] Data preprocessing: First, extract video frames at a rate of one frame per second. Visually inspect the extracted photos and filter out low-quality images such as blurry, overly bright, or overly dark images.

[0051] Initial intrinsic parameter estimation: Initial intrinsic parameters are calibrated using the OpenCV fisheye camera calibration module. OpenCV uses the KB model for its fisheye camera, and after calibration, an 8D vector is obtained representing the intrinsic parameters—[f x ,f y ,c x ,c y [k1,k2,k3,k4], corresponding to focal length, principal point, and distortion parameters.

[0052] (2) Three-dimensional calibration field:

[0053] Data collection for the calibration field: Layout of the indoor calibration field as follows Figure 2 As shown, target points are affixed to the left wall, front wall, right wall, ground, and pillars in the lens, ensuring that the target points are evenly distributed and not coplanar. To capture the image, simply place the device on a table and take one photograph.

[0054] AprilTags control point identification: AprilTags are affixed to the walls and pillars of the calibration field. The program can automatically identify the tags to obtain the control point number and corresponding 3D-2D information for subsequent calibration.

[0055] Combine the initial intrinsic parameters and the corresponding 2D / 3D point information to perform control point distortion correction: This step is used to correct the distortion of the 2D coordinates corresponding to the target points by combining the initial intrinsic parameters obtained in the previous steps, so that the initial extrinsic parameters can be calculated using the RQ method later.

[0056] The distortion correction process includes:

[0057] Normalized coordinates: ;

[0058] Among them, u d v d c represents the image pixel coordinates. x c y It is like the principal point, f x and f y It is the focal length;

[0059] Calculate the distortion radius: ;

[0060] The true angle of incidence θ is solved using Newton's iteration method:

[0061] ;

[0062] It is the true angle of incidence with added distortion information;

[0063] Iterate until convergence.

[0064] Calculate the distortion-free coordinates: ;

[0065] Output normalized distortion-free points .

[0066] (3) SVD estimation of the P matrix:

[0067] Treating the projection matrix L as a typical The vector formed by expanding the camera projection matrix by rows:

[0068]

[0069] Given the 3D coordinates of the target point and the distorted 2D coordinates, the P matrix can be solved using the SVD decomposition method.

[0070] (4) RQ decomposition to estimate the extrinsic parameters of the calibration field camera:

[0071] The P matrix can be decomposed into a... Multiply the intrinsic parameter matrix by a The extrinsic parameter matrix can be further decomposed using RQ decomposition to obtain the corresponding intrinsic and extrinsic parameters. Specifically:

[0072] Main point: ;

[0073] focal length:

[0074] ;

[0075] f is the average focal length;

[0076] , It is the camera optical axis direction obtained from RQ decomposition;

[0077] Construct R:

[0078] ;

[0079] j indicates the nth item;

[0080] The rotation matrix R is calculated using the above formula, where R is a... Matrix (r) 11 ,r 12 ,r 13 ,r 21 ,r 22 ,r 23 , r 31 ,r 32 ,r 33 );

[0081] Intrinsic parameter matrix A: ;

[0082] Camera Center U: Among them, Xs and Ys 、 Zs represents the three components at the center of the camera.

[0083] (5) Optimization of camera intrinsic and extrinsic parameter estimation based on MEI Camera model:

[0084] The projection method of the Mei model is: ;

[0085] X n Y n Theoretically, it refers to the point from a 3D point to the normalized image plane.

[0086] Where (X,Y,Z) are the normalized 3D point coordinates in the camera coordinate system;

[0087] It is the radial distance in the 3D plane;

[0088] These are the correction parameters for the Mei model, used to adjust the nonlinear distortion of the fisheye lens.

[0089] The projection process from 3D points to 2D points using a fisheye lens based on the Mei model is shown below. A loss function can be constructed based on this projection process, and an optimization algorithm can be used for optimization. The calculation process is as follows:

[0090] 1. Calculate the incident angle θ

[0091]

[0092] here It is the radial distance of the 3D normalized points.

[0093] It is the corrected depth to ensure the perspective is correct.

[0094] 2. Calculate the normalized image plane radius.

[0095] ;

[0096] This formula transforms spherical coordinates to the normalized image plane, converting the radial angle θ (i.e., the angle of incidence) in 3D space into a normalized radius. .

[0097] 3. Calculate the final normalized pixel coordinates

[0098]

[0099] This calculation method ensures that the angle of the point remains unchanged, but adapts to the deformation of fisheye lenses.

[0100] Given the projection process, construct the bundle adjustment loss function and optimize it using the Ceres optimization algorithm. The optimization terms include the principal point c. x ,c y Focal length f x ,f y Four distortion parameters k1, k2, k3, k4, and 6D exterior orientation elements [x, y, z, ω, φ, κ].

[0101] (6) Predict control points based on the model:

[0102] Having completed the above steps, we have obtained the intrinsic and extrinsic parameters required for the projection model. In this step, we input the 3D coordinates of the crosshair target in the calibration field, substitute them into the model, and calculate to obtain the two-dimensional coordinates on the image.

[0103] (7) Check and adjust the locations and re-optimize:

[0104] This step combines image information to adjust the coordinates of the 2D projection points predicted in the previous step and re-optimizes the model parameters.

[0105] (8) Output calibration file: Organize and output calibration file.

[0106] Example 2:

[0107] A fisheye lens calibration system that combines two-dimensional and three-dimensional information includes:

[0108] The two-dimensional checkerboard calibration module is used to acquire multi-angle video images of the two-dimensional checkerboard calibration board, extract video frames and filter them, and use a fisheye camera model to perform initial intrinsic parameter calibration on the filtered images.

[0109] The 3D calibration field module is used to acquire images of multiple target points set up in the 3D calibration field, identify the AprilTag labels, and automatically identify the labels to obtain the 3D-2D correspondence information of the control points.

[0110] The distortion removal module is used to combine the 2D coordinates of the initial internal control points and remove distortion through an iterative method to obtain normalized distortion-free coordinates.

[0111] The extrinsic parameter initialization module is used to solve the projection matrix P by using the distortion-free 2D points and the known 3D points, and then obtain the initial extrinsic parameters of the camera in the calibration field by RQ decomposition.

[0112] The joint optimization module is used to construct a bundle adjustment model, which integrates the observation data of the two-dimensional checkerboard and the three-dimensional calibration field, and performs global optimization of camera intrinsic parameters, distortion parameters and exterior orientation elements.

[0113] The model prediction and adjustment module is used to adjust the model parameters based on the projection positions of the control points on the image predicted by the optimized model, combined with image information.

[0114] The output calibration results module is used to generate the final camera calibration parameter file.

[0115] Example 3:

[0116] An electronic device includes a processor and a memory communicatively connected to the processor and used to store executable instructions of the processor, the processor being used to execute the aforementioned fisheye lens calibration method combining two-dimensional and three-dimensional information.

[0117] Example 4:

[0118] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the aforementioned fisheye lens calibration method that combines two-dimensional and three-dimensional information.

[0119] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy and procedures.

[0121] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0122] The acquisition, transmission, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.

[0123] It should be noted that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary and are intended only to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used such solutions.

[0124] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0126] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0127] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). In addition, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning paper or other media, followed by editing, interpreting or otherwise processing as necessary, and then stored in computer memory.

[0128] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0129] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0130] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0131] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for calibrating a fisheye lens by combining two-dimensional and three-dimensional information, characterized in that: Includes the following steps: Two-dimensional checkerboard calibration stage: acquire multi-angle video images of the two-dimensional checkerboard calibration board, extract and filter video frames, and use a fisheye camera model to perform initial intrinsic parameter calibration on the filtered images; 3D calibration field stage: Collect images of multiple target points set up in the 3D calibration field, identify the AprilTag labels, and automatically identify the labels to obtain the 3D-2D correspondence information of the control points; Distortion removal: Combining the 2D coordinates of the initial intrinsic control points, distortion is removed through an iterative method to obtain normalized distortion-free coordinates; Extrinsic parameter initialization: Using the distortion-free 2D points and known 3D points, the projection matrix P is solved by SVD decomposition, and then the initial extrinsic parameters of the camera in the calibration field are obtained by RQ decomposition. Joint optimization: Construct a bundle adjustment model, fuse the observation data of the two-dimensional checkerboard and the three-dimensional calibration field, and perform global optimization of camera intrinsic parameters, distortion parameters and exterior orientation elements; Model prediction and adjustment: Based on the predicted projection positions of control points on the image after optimization, adjustments are made in conjunction with image information, and model parameters are re-optimized; Output calibration results: Generate the final camera calibration parameter file.

2. The fisheye lens calibration method combining two-dimensional and three-dimensional information according to claim 1, characterized in that: The distortion correction process specifically includes: Convert the distorted image coordinates to normalized coordinates; Calculate the distortion radius; The true angle of incidence is determined using Newton's iterative method. Calculate the normalized coordinates after distortion removal.

3. The fisheye lens calibration method combining two-dimensional and three-dimensional information according to claim 1, characterized in that: In the external parameter initialization step: Solve the 3×4 projection matrix P using SVD decomposition; The initial exterior orientation elements are obtained by decomposing the P matrix into intrinsic and extrinsic parameter matrices through RQ decomposition.

4. The fisheye lens calibration method combining two-dimensional and three-dimensional information according to claim 1, characterized in that: In the joint optimization step: Construct the bundle adjustment loss function; Ceres was used to optimize the principal point, focal length, distortion parameters, and exterior orientation elements.

5. The fisheye lens calibration method combining two-dimensional and three-dimensional information according to claim 1, characterized in that: During the three-dimensional calibration stage, the indoor calibration field layout must meet the following conditions: The target points are distributed across multiple non-coplanar spatial planes, including walls, floors, and cylindrical surfaces; The target point is the AprilTag tag, which supports automatic recognition and number matching.

6. The fisheye lens calibration method combining two-dimensional and three-dimensional information according to claim 1, characterized in that: In the initial intrinsic parameter calibration of the selected images using a fisheye camera model, the OpenCV fisheye camera calibration module is used to calibrate the initial intrinsic parameters. The fisheye camera model used by OpenCV is the KB model. After calibration, an 8D vector is obtained to represent the intrinsic parameters, including focal length, principal point, and distortion parameters.

7. A fisheye lens calibration system that combines two-dimensional and three-dimensional information, characterized in that: include: The two-dimensional checkerboard calibration module is used to acquire multi-angle video images of the two-dimensional checkerboard calibration board, extract video frames and filter them, and use a fisheye camera model to perform initial intrinsic parameter calibration on the filtered images. The 3D calibration field module is used to acquire images of multiple target points set up in the 3D calibration field, identify the AprilTag labels, and automatically identify the labels to obtain the 3D-2D correspondence information of the control points. The distortion removal module is used to combine the 2D coordinates of the initial internal control points and remove distortion through an iterative method to obtain normalized distortion-free coordinates. The extrinsic parameter initialization module is used to solve the projection matrix P by using the distortion-free 2D points and the known 3D points, and then obtain the initial extrinsic parameters of the camera in the calibration field by RQ decomposition. The joint optimization module is used to construct a bundle adjustment model, which integrates the observation data of the two-dimensional checkerboard and the three-dimensional calibration field, and performs global optimization of camera intrinsic parameters, distortion parameters and exterior orientation elements. The model prediction and adjustment module is used to adjust the model parameters based on the projection positions of the control points on the image predicted by the optimized model, combined with image information. The output calibration results module is used to generate the final camera calibration parameter file.

8. An electronic device comprising a processor and a memory communicatively connected to the processor and used for storing processor-executable instructions, characterized in that: The processor is used to execute the method described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images