Stereo target based camera intrinsic parameter calibration method and apparatus

Abstract

The application relates to a camera internal parameter calibration method and device based on a stereo target. The method comprises the following steps: taking a stereo target by a camera to be calibrated to obtain a calibration image; the stereo target is provided with coded points and non-coded points; based on the calibration image, pixel coordinates of the coded points and the non-coded points are extracted, and the pixel coordinates are matched with physical coordinates to calculate initial internal parameters and initial external parameters; and the initial internal parameters and the initial external parameters are iteratively optimized to obtain target internal parameters and target external parameters. By using the method, only one calibration image is taken, the camera internal parameters are quickly and accurately calibrated, and high-precision camera internal parameters are obtained.

Description

Technical Field

[0001] This application relates to the field of camera technology, and in particular to a method and apparatus for calibrating camera intrinsic parameters based on a stereo target. Background Technology

[0002] Camera intrinsic parameter calibration is a technique that solves for camera intrinsic parameters through the correspondence between object and image space. The accuracy of the calibration results directly affects the accuracy of the results produced by the camera. With the development of technologies such as autonomous driving, applications such as binocular ranging, 3D reconstruction, high-precision positioning and mapping, and surround view systems all rely on cameras, which also places higher demands on the calibration of camera intrinsic parameters.

[0003] The most widely used camera calibration technology is based on Zhang's calibration method. The camera adjusts its pose and takes multiple pictures of a specified planar calibration board (chessboard, array circle) to obtain the coordinates of the feature points of the calibration board and calculate the camera's intrinsic and extrinsic parameters.

[0004] However, the above method requires taking multiple images of the target plate, which is cumbersome, time-consuming, and has low calibration accuracy. Summary of the Invention

[0005] Therefore, it is necessary to provide a camera intrinsic parameter calibration method and apparatus based on a stereo target that can improve calibration accuracy and speed, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a camera intrinsic parameter calibration method based on a stereo target, the method comprising:

[0007] A calibration image is obtained by photographing a stereo target using a camera to be calibrated; wherein the stereo target has coded points and non-coded points; based on the calibration image, the first pixel coordinates of the coded points and the second pixel coordinates of the non-coded points are extracted;

[0008] Obtain the three-dimensional reconstruction information of the stereo target, the three-dimensional reconstruction information including the first physical coordinates of the coded points and the second physical coordinates of the non-coded points; identify the coded value of the coded point from the calibration image, and obtain the first physical coordinates that match the first pixel coordinates based on the coded value;

[0009] Obtain the initial intrinsic parameters of the camera to be calibrated; calculate the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates, and the first physical coordinates after matching; project the second physical coordinates based on the initial intrinsic parameters and the initial extrinsic parameters to obtain the projected pixel coordinates of the non-coded point;

[0010] Based on the matching result of the second pixel coordinates and the projected pixel coordinates, the initial intrinsic parameters and the initial extrinsic parameters are optimized and iterated to obtain the target intrinsic parameters and target extrinsic parameters of the camera to be calibrated.

[0011] In one embodiment, obtaining the three-dimensional reconstruction information of the stereo target, wherein the three-dimensional reconstruction information includes the first physical coordinates of the coded points and the second physical coordinates of the non-coded points, includes:

[0012] A stereo target is photographed using a camera with known intrinsic parameters to obtain reconstructed images in different poses, wherein at least 200 reconstructed images are obtained;

[0013] Based on the reconstructed image, the coded points and the non-coded points are reconstructed in three dimensions to obtain the first physical coordinates of the coded points and the second physical coordinates of the non-coded points.

[0014] In one embodiment, obtaining the initial intrinsic parameters of the camera to be calibrated includes:

[0015] Initial intrinsic parameters are calculated based on the design information of the camera to be calibrated, which includes camera specifications, lens specifications, image sensor specifications, and lens image height table.

[0016] In one embodiment, the calculation of the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates, and the matched first physical coordinates includes:

[0017] Based on the initial intrinsic parameters, the encoding point is subjected to distortion correction processing to obtain the first pixel coordinates after distortion correction;

[0018] The initial extrinsic parameters are calculated based on the initial intrinsic parameters, the first pixel coordinates after distortion correction, and the first physical coordinates after matching.

[0019] In one embodiment, the step of optimizing and iterating the initial intrinsic and extrinsic parameters based on the matching result of the second pixel coordinates and the projected pixel coordinates to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated includes:

[0020] The second pixel coordinates and the projected pixel coordinates are matched to obtain the successfully matched non-coded point pair;

[0021] Based on the successfully matched non-coded point pairs, the initial intrinsic and extrinsic parameters are optimized and iterated to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated.

[0022] In one embodiment, matching the second pixel coordinates and the projected pixel coordinates to obtain the successfully matched non-coded point pair includes:

[0023] Calculate the Euclidean distance between the second pixel coordinates and the projected pixel coordinates;

[0024] When the calculated Euclidean distance is less than the preset distance, the second pixel coordinate and the projected pixel coordinate are successfully matched, and the successfully matched non-coded point pair is obtained.

[0025] In one embodiment, the optimization iteration of the initial intrinsic and extrinsic parameters based on the successfully matched non-coded point pairs to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated includes:

[0026] Based on the successfully matched non-coded point pairs, the initial intrinsic parameters and the initial extrinsic parameters are optimized iteratively to obtain intermediate intrinsic parameters and intermediate extrinsic parameters.

[0027] The non-coded points that were successfully matched were reprojected based on intermediate intrinsic and extrinsic parameters, and the reprojection error was calculated.

[0028] Based on the reprojection error, the non-coded point pairs with incorrect matching are removed, and the non-coded point pairs with correct matching are obtained;

[0029] Based on the correctly matched non-coded point pairs, the target intrinsic and target extrinsic parameters of the camera to be calibrated are obtained.

[0030] Secondly, this application also provides a camera intrinsic parameter calibration device based on a stereo target, the device comprising:

[0031] A two-dimensional information acquisition module is used to capture a stereo target using a camera to be calibrated to obtain a calibration image; wherein, the stereo target is provided with coded points and non-coded points; based on the calibration image, the first pixel coordinates of the coded points and the second pixel coordinates of the non-coded points are extracted;

[0032] A 3D information acquisition module is used to acquire the 3D reconstruction information of the stereo target, wherein the 3D reconstruction information includes the first physical coordinates of the coded points and the second physical coordinates of the non-coded points; the coded value of the coded point is identified from the calibration image, and the first physical coordinates matching the first pixel coordinates are obtained based on the coded value;

[0033] The initial parameter calculation module is used to obtain the initial intrinsic parameters of the camera to be calibrated; based on the initial intrinsic parameters, the first pixel coordinates, and the first physical coordinates after matching, the initial extrinsic parameters are calculated; based on the initial intrinsic parameters and the initial extrinsic parameters, the second physical coordinates are projected to obtain the projected pixel coordinates of the non-coded point;

[0034] The target parameter calculation module is used to optimize and iterate the initial intrinsic parameters and the initial extrinsic parameters based on the matching result of the second pixel coordinates and the projected pixel coordinates, so as to obtain the target intrinsic parameters and target extrinsic parameters of the camera to be calibrated.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the above-described methods for calibrating camera intrinsic parameters based on a stereo target.

[0036] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of any of the above-described methods for calibrating camera intrinsic parameters based on a stereo target.

[0037] The aforementioned camera intrinsic parameter calibration method and apparatus based on stereo targets captures calibration images of stereo targets with known 3D reconstruction information. Two-dimensional information of coded and non-coded points is extracted from the calibration images and matched with the 3D information. First, extrinsic parameters are calculated using the matched coded points. Then, the calculated extrinsic parameters are used to match the non-coded points, thereby optimizing and iterating the intrinsic and extrinsic parameters to obtain the target intrinsic and extrinsic parameters. This method enables rapid and accurate calibration of camera intrinsic parameters based solely on a single calibration image captured by the camera to be calibrated, yielding high-precision camera intrinsic parameters. Attached Figure Description

[0038] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is a hardware structure block diagram of a camera intrinsic parameter calibration method based on a stereo target in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a camera intrinsic parameter calibration method based on a stereo target in one embodiment.

[0041] Figure 3 This is a schematic diagram of the coding points in one embodiment;

[0042] Figure 4 This is a flowchart illustrating a preferred embodiment of a camera intrinsic parameter calibration method based on a stereo target.

[0043] Figure 5 This is a structural block diagram of a camera intrinsic parameter calibration device based on a stereo target in one embodiment;

[0044] Figure 6 This is a schematic diagram of the structure of a three-dimensional target in one embodiment.

[0045] Reference numerals: 10, frame; 20, planar plate; 30, camera mounting bracket. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0047] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of this application.

[0048] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the camera intrinsic parameter calibration method based on a stereo target in this embodiment. Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0051] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the camera intrinsic parameter calibration method based on a stereo target in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0053] This embodiment provides a camera intrinsic parameter calibration method based on a stereo target. Figure 2 This is a flowchart illustrating the camera intrinsic parameter calibration method based on a stereo target in this embodiment. Figure 2 As shown, the process includes the following steps:

[0054] Step S210: Use the camera to be calibrated to capture a stereo target to obtain a calibration image; wherein, the stereo target has coded points and non-coded points; based on the calibration image, extract the first pixel coordinates of the coded points and the second pixel coordinates of the non-coded points.

[0055] Specifically, a camera bracket is provided in front of the stereo target. The camera to be calibrated is mounted in a fixture on the camera bracket to capture an image of the stereo target, so that the obtained calibration image covers the entire field of view (FOV) of the camera to be calibrated.

[0056] Step S220: Obtain the three-dimensional reconstruction information of the stereo target, which includes the first physical coordinates of the coded points and the second physical coordinates of the non-coded points; identify the coded values ​​of the coded points from the calibration image, and obtain the first physical coordinates that match the first pixel coordinates based on the coded values.

[0057] Specifically, the known reconstruction results of the 3D target are obtained by performing a 3D reconstruction of the 3D target to acquire the physical coordinates of each coded and non-coded point. The first and second physical coordinates are the coordinate values ​​of each point in the world coordinate system.

[0058] Each coded point is a binary coded pattern with a unique coded value. The coded point consists of a solid circle and a ring; see [link to relevant documentation]. Figure 3 The ring is divided into segments and marked with two colors, such as black and white. Through binary encoding, a unique code value can be calculated for each code point.

[0059] Step S230: Obtain the initial intrinsic parameters of the camera to be calibrated; calculate the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates, and the first physical coordinates after matching; project the second physical coordinates based on the initial intrinsic parameters and the initial extrinsic parameters to obtain the projected pixel coordinates of the non-coded points.

[0060] Specifically, the initial intrinsic parameters can be calculated based on the design information of the camera to be calibrated, including camera specifications, lens specifications, image sensor specifications, and lens image height tables; alternatively, they can be obtained from pre-stored initial intrinsic parameters in a database, or calculated using other intrinsic parameter calibration methods. The accuracy of the initial intrinsic parameters is relatively low, requiring further optimization of the calculation.

[0061] Specifically, distortion correction can be performed on the encoded points based on the initial intrinsic parameters to obtain the first pixel coordinates after distortion correction; the initial extrinsic parameters can be calculated based on the initial intrinsic parameters, the first pixel coordinates after distortion correction, and the first physical coordinates after matching.

[0062] Step S240: Based on the matching result of the second pixel coordinates and the projected pixel coordinates, the initial intrinsic and extrinsic parameters are optimized and iterated to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated.

[0063] Specifically, the Euclidean distance between the second pixel coordinates and the projected pixel coordinates is calculated. When the calculated Euclidean distance is less than a preset distance, the second pixel coordinates and the projected pixel coordinates are considered to be successfully matched, resulting in a successfully matched non-coded point pair. Based on the successfully matched non-coded point pairs, the initial intrinsic and extrinsic parameters are optimized iteratively to obtain intermediate intrinsic and extrinsic parameters. Based on the intermediate intrinsic and extrinsic parameters, the successfully matched non-coded points are reprojected, and the reprojection error is calculated. Based on the reprojection error, incorrectly matched non-coded point pairs are removed, resulting in correctly matched non-coded point pairs. Based on the correctly matched non-coded point pairs, the target intrinsic and extrinsic parameters of the camera to be calibrated are obtained.

[0064] In the aforementioned camera intrinsic parameter calibration method based on stereo targets, calibration images are captured on stereo targets with known 3D reconstruction information. Two-dimensional information of coded and non-coded points is extracted from the calibration images and matched with the 3D information. First, the extrinsic parameters are calculated using the matched coded points. Then, the calculated extrinsic parameters are used to match the non-coded points. The intrinsic and extrinsic parameters are then optimized and iterated to obtain the target intrinsic and extrinsic parameters. This method can quickly and accurately calibrate camera intrinsic parameters based on only one calibration image captured by the camera to be calibrated, thus obtaining high-precision camera intrinsic parameters.

[0065] In one embodiment, based on the above step S220, the three-dimensional reconstruction information of the stereo target is obtained. The three-dimensional reconstruction information includes the first physical coordinates of the encoded points and the second physical coordinates of the non-coded points, which may specifically include the following steps:

[0066] Step S221: Use a camera with known intrinsic parameters to photograph a stereo target to obtain reconstructed images in different poses, with at least 200 reconstructed images.

[0067] Among them, the camera with known intrinsic parameters is the camera with known high-precision intrinsic parameters.

[0068] Step S222: Perform three-dimensional reconstruction of coded points and non-coded points based on the reconstructed image to obtain the first physical coordinates of the coded points and the second physical coordinates of the non-coded points.

[0069] Specifically, based on Structure From Motion (SFM), 3D reconstruction is performed on coded and non-coded points to obtain the first and second physical coordinates in 3D. The first physical coordinates are then stored in correspondence with the coded values ​​of the coded points. Alternatively, a specified coding sequence can be assigned to each non-coded point, and the second physical coordinates can be stored in correspondence with the coding sequence for easy statistical analysis of non-coded points.

[0070] This embodiment allows for the acquisition of accurate 3D information on coded and non-coded points using a camera with known parameters, thereby improving the calibration accuracy and speed of the camera parameters to be calibrated.

[0071] In one embodiment, obtaining the initial intrinsic parameters of the camera to be calibrated based on step S230 described above may specifically include the following steps:

[0072] Step S231: Calculate initial intrinsic parameters based on the design information of the camera to be calibrated. The design information includes camera specifications, lens specifications, image sensor specifications, and lens image height table.

[0073] Specifically, the initial intrinsic parameters include the initial pixel focal length, initial principal pixel coordinates, and initial distortion coefficients. The initial pixel focal length and initial principal pixel coordinates are obtained based on the camera specifications, lens specifications, and image sensor specifications; the initial distortion coefficients are obtained by fitting a lens image height table. The initial intrinsic parameters can be calculated offline.

[0074] Through this embodiment, relatively accurate initial intrinsic parameters of the camera to be calibrated can be obtained.

[0075] In one embodiment, based on step S230 above, the initial extrinsic parameters are calculated based on the initial intrinsic parameters, the first pixel coordinates, and the matched first physical coordinates. This may specifically include the following steps:

[0076] Step S232: Perform distortion correction on the encoded points based on the initial intrinsic parameters to obtain the first pixel coordinates after distortion correction.

[0077] Step S233: Calculate the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates after distortion removal, and the first physical coordinates after matching.

[0078] Specifically, the extrinsic parameters are solved using the PnP method, and the calculation formula is as follows:

[0079]

[0080] Among them, [u,v] T Let X be the coordinates of the first pixel after distortion correction, and K be the initial intrinsic parameters of the camera. w ,Y w Z w ] T R is the first physical coordinate of the encoding point. 3×3 and T 3×1 These are the initial external parameters.

[0081] Through this embodiment, the initial extrinsic parameters of the camera coordinate system to be calibrated relative to the three-dimensional target coordinate system can be obtained.

[0082] In one embodiment, based on step S240 above, and based on the matching result of the second pixel coordinates and the projected pixel coordinates, the initial intrinsic and initial extrinsic parameters are optimized iteratively to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated. Specifically, this may include the following steps:

[0083] Step S241: Match the second pixel coordinates and the projected pixel coordinates to obtain a successfully matched non-coded point pair.

[0084] Specifically, the projected pixel coordinates are calculated from the second physical coordinates, which contain the coordinate information of all non-coded points on the stereo target. The second pixel coordinates are detected from the calibration image. Due to the shooting angle, the second pixel coordinates only contain the coordinate information of some non-coded points. Therefore, the second pixel coordinates and the projected pixel coordinates are matched, the undetected non-coded points are removed, and the second physical coordinates and second pixel coordinates of the successfully matched non-coded points are retained to obtain the successfully matched non-coded point pairs.

[0085] Specifically, step S241 can be implemented by steps S241a to S241b:

[0086] Step S241a: Calculate the Euclidean distance between the second pixel coordinates and the projected pixel coordinates.

[0087] In step S241b, when the calculated Euclidean distance is less than the preset distance, the second pixel coordinates and the projected pixel coordinates are successfully matched, resulting in a successfully matched non-coded point pair.

[0088] Step S242: Based on the successfully matched non-coded point pairs, optimize and iterate the initial intrinsic and extrinsic parameters to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated.

[0089] Specifically, the error between the projected pixel coordinates and the second pixel coordinates is calculated. A loss function is established based on the errors of all non-coded points. After optimizing the intrinsic and extrinsic parameters, the projected coordinates of the non-coded points are calculated using the adjusted intrinsic and extrinsic parameters. The error between the projected coordinates and the pixel coordinates is also calculated. The loss function is calculated. The above steps of projection and error calculation are repeated to optimize the loss function result until the loss function reaches its minimum value. The current intrinsic and extrinsic parameters are the optimal solution, which are the target intrinsic and extrinsic parameters.

[0090] In this embodiment, by matching non-coding, the intrinsic and extrinsic parameters of the camera are optimized and iterated to obtain the optimal target intrinsic and extrinsic parameters, thereby improving the calculation accuracy of camera parameters.

[0091] In one embodiment, based on step S242 above, and based on the successfully matched non-coded point pairs, the initial intrinsic and extrinsic parameters are optimized iteratively to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated, including:

[0092] Step S242a: Based on the successfully matched non-coded point pairs, optimize and iterate the initial intrinsic and extrinsic parameters to obtain intermediate intrinsic and extrinsic parameters.

[0093] Step S242b: Reproject the successfully matched non-coded points based on the intermediate intrinsic and extrinsic parameters, and calculate the reprojection error.

[0094] Among them, the reprojection of the second physical coordinates of the successfully matched non-coded points can be performed using the root mean square (RMS) to calculate the reprojection error.

[0095] Step S242c: Based on the reprojection error, remove non-coded points with incorrect matching to obtain the correct non-coded points.

[0096] Specifically, the non-coded point pairs with mismatches mainly include mismatches caused by occlusion and non-coded point pairs with low detection accuracy. The correctness of the matching of non-coded point pairs is judged based on the reprojection error. If there are mismatches, the mismatched point pairs are removed, and the calibration step of optimization iteration is re-executed in step S242a.

[0097] Step S242d: Based on the correctly matched non-coded point pairs, the target intrinsic and extrinsic parameters of the camera to be calibrated are obtained.

[0098] In this embodiment, by utilizing reprojection error optimization, non-coded point pairs that are incorrectly matched in the batch are eliminated, thereby improving the accuracy of the optimization iteration results.

[0099] The present embodiment will now be described and illustrated through preferred embodiments.

[0100] Figure 4 This is a flowchart of a camera intrinsic parameter calibration method based on a stereo target according to a preferred embodiment of this invention.

[0101] Step S401: Use the camera to be calibrated to photograph the stereo target to obtain a calibration image; wherein, the stereo target has coded points and non-coded points.

[0102] Step S402: Based on the calibration image, extract the first pixel coordinates of the coded point and identify the coded value of the coded point from the calibration image; obtain the first physical coordinates of the coded point, and based on the coded value, obtain the first physical coordinates that match the first pixel coordinates.

[0103] Step S403: Based on the calibration image, extract the second pixel coordinates of the non-coded points.

[0104] Step S404: Calculate the initial intrinsic parameters based on the camera design specifications and lens image height table of the camera to be calibrated.

[0105] Step S405: Perform distortion correction on the encoded points based on the initial intrinsic parameters to obtain the first pixel coordinates after distortion correction.

[0106] Step S406: Calculate the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates after distortion removal, and the first physical coordinates after matching.

[0107] Step S407: Obtain the second physical coordinates of the non-coded points; project the second physical coordinates based on the initial intrinsic and extrinsic parameters to obtain the projected pixel coordinates of the non-coded points; match the second pixel coordinates and the projected pixel coordinates to obtain a successfully matched pair of non-coded points.

[0108] Step S408: Based on the successfully matched non-coded point pairs, optimize and iterate the initial intrinsic and extrinsic parameters to obtain intermediate intrinsic and extrinsic parameters.

[0109] Step S409: Reproject the successfully matched non-coded point pairs based on the intermediate intrinsic and extrinsic parameters, and calculate the reprojection error; based on the reprojection error, determine whether there are any incorrectly matched non-coded point pairs.

[0110] In step S410, if there is a non-coded point pair with a mismatch, the mismatched non-coded point pair is removed, and steps S408 to S410 are executed again; if there is no mismatched non-coded point pair, a correctly matched non-coded point pair is obtained, and step S411 is executed.

[0111] Step S411: Based on the correctly matched non-coded point pairs, obtain the target intrinsic and target extrinsic parameters of the camera to be calibrated.

[0112] In this preferred embodiment, a calibration image is captured on a stereo target with known 3D reconstruction information. Two-dimensional information of coded and non-coded points is extracted from the calibration image and matched with the 3D information. First, the extrinsic parameters are calculated using the matched coded points. Then, the calculated extrinsic parameters are used to match non-coded point pairs. The intrinsic and extrinsic parameters are then optimized and iterated to obtain the target intrinsic and extrinsic parameters. This allows for fast and accurate calibration of camera intrinsic parameters based on only one calibration image captured by the camera to be calibrated, thus obtaining high-precision camera intrinsic parameters.

[0113] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0114] Based on the same inventive concept, this embodiment also provides a camera intrinsic parameter calibration device based on a stereo target. This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0115] In one embodiment, such as Figure 5 As shown, a camera intrinsic parameter calibration device based on a stereo target is provided, including: a two-dimensional information acquisition module 51, a three-dimensional information acquisition module 52, an initial parameter calculation module 53, and a target parameter calculation module 54, wherein:

[0116] The two-dimensional information acquisition module 51 is used to capture a stereo target using the camera to be calibrated to obtain a calibration image; wherein, the stereo target is provided with coded points and non-coded points; based on the calibration image, the first pixel coordinates of the coded points and the second pixel coordinates of the non-coded points are extracted;

[0117] The 3D information acquisition module 52 is used to acquire the 3D reconstruction information of the stereo target. The 3D reconstruction information includes the first physical coordinates of the coded points and the second physical coordinates of the non-coded points. The coded values ​​of the coded points are identified from the calibration image, and the first physical coordinates that match the first pixel coordinates are obtained based on the coded values.

[0118] The initial parameter calculation module 53 is used to obtain the initial intrinsic parameters of the camera to be calibrated; based on the initial intrinsic parameters, the first pixel coordinates, and the first physical coordinates after matching, the initial extrinsic parameters are calculated; based on the initial intrinsic parameters and the initial extrinsic parameters, the second physical coordinates are projected to obtain the projected pixel coordinates of the non-coded points;

[0119] The target parameter calculation module 54 is used to optimize and iterate the initial intrinsic and extrinsic parameters based on the matching results of the second pixel coordinates and the projected pixel coordinates, so as to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated.

[0120] In one embodiment, the 3D information acquisition module 52 is further configured to: capture a stereo target using a camera with known intrinsic parameters to obtain reconstructed images in different poses, wherein the reconstructed images are at least 200; and perform 3D reconstruction of coded points and non-coded points based on the reconstructed images to obtain the first physical coordinates of the coded points and the second physical coordinates of the non-coded points.

[0121] In one embodiment, the initial parameter calculation module 53 is further configured to calculate initial intrinsic parameters based on the design information of the camera to be calibrated, including camera specifications, lens specifications, image sensor specifications, and lens image height table.

[0122] In one embodiment, the initial parameter calculation module 53 is further configured to: perform distortion correction on the encoded point based on the initial intrinsic parameters to obtain the first pixel coordinates after distortion correction; and calculate the initial extrinsic parameters based on the initial intrinsic parameters, the first pixel coordinates after distortion correction, and the first physical coordinates after matching.

[0123] In one embodiment, the target parameter calculation module 54 is further configured to match the second pixel coordinates and the projected pixel coordinates to obtain a successfully matched non-coded point pair; and based on the successfully matched non-coded point pair, optimize and iterate the initial intrinsic and extrinsic parameters to obtain the target intrinsic and extrinsic parameters of the camera to be calibrated.

[0124] In one embodiment, the target parameter calculation module 54 is further configured to calculate the Euclidean distance between the second pixel coordinates and the projected pixel coordinates; when the calculated Euclidean distance is less than a preset distance, the second pixel coordinates and the projected pixel coordinates are successfully matched, and a successfully matched non-coded point pair is obtained.

[0125] In one embodiment, the target parameter calculation module 54 is further configured to: optimize and iterate the initial intrinsic and extrinsic parameters based on the successfully matched non-coded point pairs to obtain intermediate intrinsic and extrinsic parameters; reproject the successfully matched non-coded points based on the intermediate intrinsic and extrinsic parameters and calculate the reprojection error; remove the incorrectly matched non-coded point pairs based on the reprojection error to obtain correctly matched non-coded point pairs; and obtain the target intrinsic and extrinsic parameters of the camera to be calibrated based on the correctly matched non-coded point pairs.

[0126] Each module in the aforementioned camera intrinsic parameter calibration device based on a stereo target can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0127] This embodiment also provides a stereo target, which is applicable to any of the above-described camera intrinsic parameter calibration methods based on stereo targets. See also Figure 6 The three-dimensional target includes a frame 10 and at least two non-coplanar planar target plates 20, which are mounted on the frame 10. Each planar target plate 20 has coded points and non-coded points. The coded points are composed of solid circles and rings and have unique codes. The non-coded points are the intersections of a checkerboard pattern.

[0128] Furthermore, the frame 10 includes a calibration plate frame and a calibration plate support, with a gap between the calibration plate frame and the calibration plate support; the planar calibration plates 20 consist of four pieces, including a first planar calibration plate 20, a second planar calibration plate 20, a third planar calibration plate 20 and a fourth planar calibration plate 20, wherein the second planar calibration plate 20, the third planar calibration plate 20 and the fourth planar calibration plate 20 are fixedly installed in the calibration plate frame, and the first planar calibration plate 20 is installed on the calibration plate support.

[0129] Furthermore, the frame 10 is made of steel, and the four flat standard plates 20 are fixed to the calibration plate frame or calibration plate bracket using a threaded connection.

[0130] Furthermore, the distance between the first planar marker 20 and the fourth planar marker 20 is 100mm.

[0131] Furthermore, the second planar marker 20 forms angles with the third planar marker 20 and the fourth planar marker 20, respectively, with angles of 45° and 135°.

[0132] Furthermore, the frame 10 is equipped with four independent light sources to provide supplementary lighting for the four planar target plates 20.

[0133] Furthermore, the material for the flat label 20 is film.

[0134] Furthermore, the dimensions of the four flat standard plates 20 are 1100mm×1500mm×30mm, 800mm×1500mm×30mm, 800mm×1500mm×30mm, and 400mm×500mm×30mm.

[0135] Further, see Figure 6 The stereo target also includes a camera mounting bracket 30, which is located at the front end of the frame 10 and connected to the bottom end of the frame 10. The camera mounting bracket 30 is equipped with a fixture for placing the camera to be calibrated. The fixture is equipped with a slider to adjust the distance between the camera and the stereo target plate. The length of the camera mounting bracket 30 can be adjusted according to the actual application.

[0136] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-described methods for camera intrinsic parameter calibration based on a stereo target.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for calibrating camera intrinsic parameters based on a stereo target.

[0138] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the above-described methods for camera intrinsic parameter calibration based on a stereo target.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A stereotactic target-based camera intrinsic parameter calibration method, characterized in that, The method comprises: photographing a stereo target by a camera to be calibrated to obtain a calibration image; wherein the stereo target is provided with coded points and non-coded points; based on the calibration image, first pixel coordinates of the coded points and second pixel coordinates of the non-coded points are extracted; photographing the stereo target by a camera with known intrinsic parameters to obtain reconstruction images in different poses, based on the reconstruction images, three-dimensional reconstruction is performed on the coded points and the non-coded points to obtain first physical coordinates of the coded points and second physical coordinates of the non-coded points; the coded values of the coded points are identified from the calibration image, based on the coded values, the first physical coordinates matched with the first pixel coordinates are obtained; obtaining initial intrinsic parameters of the camera to be calibrated; based on the initial intrinsic parameters, the first pixel coordinates and the first physical coordinates after matching, initial extrinsic parameters are calculated; based on the initial intrinsic parameters and the initial extrinsic parameters, the second physical coordinates are projected to obtain projection pixel coordinates of the non-coded points; based on the matching results of the second pixel coordinates and the projection pixel coordinates, the initial intrinsic parameters and the initial extrinsic parameters are iteratively optimized to obtain target intrinsic parameters and target extrinsic parameters of the camera to be calibrated.

2. The stereo target based camera intrinsic calibration method of claim 1, wherein, The reconstruction images are at least 200.

3. The stereo target based camera intrinsic calibration method of claim 1, wherein, The initial intrinsic parameters of the camera to be calibrated are obtained, comprising: calculating initial intrinsic parameters based on design information of the camera to be calibrated, the design information comprising camera specification, lens specification, image sensor specification and lens image height table.

4. The stereo target based camera intrinsic calibration method of claim 1, wherein, Based on the initial intrinsic parameters, the first pixel coordinates and the first physical coordinates after matching, the initial extrinsic parameters are calculated, comprising: based on the initial intrinsic parameters, the coded points are subjected to de-distortion processing to obtain de-distorted first pixel coordinates; based on the initial intrinsic parameters, the de-distorted first pixel coordinates and the first physical coordinates after matching, the initial extrinsic parameters are calculated.

5. The stereo target based camera intrinsic calibration method of claim 1, wherein, Based on the matching results of the second pixel coordinates and the projection pixel coordinates, the initial intrinsic parameters and the initial extrinsic parameters are iteratively optimized to obtain target intrinsic parameters and target extrinsic parameters of the camera to be calibrated, comprising: matching the second pixel coordinates and the projection pixel coordinates to obtain the non-coded point pairs that are successfully matched; based on the non-coded point pairs that are successfully matched, the initial intrinsic parameters and the initial extrinsic parameters are iteratively optimized to obtain target intrinsic parameters and target extrinsic parameters of the camera to be calibrated.

6. The stereo target based camera intrinsic calibration method of claim 5, wherein, The matching of the second pixel coordinates and the projection pixel coordinates to obtain the non-coded point pairs that are successfully matched, comprising: calculating the Euclidean distance of the second pixel coordinates and the projection pixel coordinates; when the calculated Euclidean distance is less than a preset distance, the second pixel coordinates and the projection pixel coordinates are successfully matched, and the non-coded point pairs that are successfully matched are obtained.

7. The stereo target based camera intrinsic calibration method of claim 5, wherein, Based on the non-coded point pairs that are successfully matched, the initial intrinsic parameters and the initial extrinsic parameters are iteratively optimized to obtain target intrinsic parameters and target extrinsic parameters of the camera to be calibrated, comprising: Based on the matched non-coding point pair, the initial intrinsic parameter and the initial extrinsic parameter are iteratively optimized to obtain an intermediate intrinsic parameter and an intermediate extrinsic parameter; Based on the intermediate intrinsic parameter and the intermediate extrinsic parameter, the matched non-coding points are re-projected, and a re-projection error is calculated; Based on the re-projection error, the non-coding point pair that is matched incorrectly is removed to obtain a non-coding point pair that is matched correctly; Based on the non-coding point pair that is matched correctly, a target intrinsic parameter and a target extrinsic parameter of the camera to be calibrated are obtained.

8. A stereotactic target-based camera intrinsic parameter calibration apparatus, comprising: The device comprises: An acquisition two-dimensional information module is configured to capture a stereo target by using a camera to be calibrated to obtain a calibration image; wherein the stereo target is provided with coding points and non-coding points; based on the calibration image, a first pixel coordinate of the coding point and a second pixel coordinate of the non-coding point are extracted; An acquisition three-dimensional information module is configured to capture a stereo target by using a camera with a known intrinsic parameter to obtain a reconstruction image in different poses, and based on the reconstruction image, the coding points and the non-coding points are three-dimensionally reconstructed to obtain a first physical coordinate of the coding point and a second physical coordinate of the non-coding point; an encoding value of the coding point is identified from the calibration image, and based on the encoding value, the first physical coordinate matched with the first pixel coordinate is obtained; An initial parameter calculation module is configured to acquire an initial intrinsic parameter of the camera to be calibrated; based on the initial intrinsic parameter, the first pixel coordinate and the first physical coordinate after matching, an initial extrinsic parameter is calculated; based on the initial intrinsic parameter and the initial extrinsic parameter, the second physical coordinate is projected to obtain a projection pixel coordinate of the non-coding point; A target parameter calculation module is configured to, based on a matching result of the second pixel coordinate and the projection pixel coordinate, iteratively optimize the initial intrinsic parameter and the initial extrinsic parameter to obtain a target intrinsic parameter and a target extrinsic parameter of the camera to be calibrated. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the camera intrinsic parameter calibration method based on a stereo target according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the camera intrinsic parameter calibration method based on a stereo target according to any one of claims 1 to 7.

Images