A dual-camera calibration processing method and device, electronic equipment and storage medium

Abstract

The application discloses a dual-camera calibration processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring each group of calibration plate images collected by the dual cameras during the movement of a high-precision control platform along a set direction at a fixed step length, and the actual ideal coordinates of the calibration points in the ideal coordinate system; according to the to-be-calibrated parameters of the dual cameras and the pixel coordinates of the calibration points in each group of calibration plate images, reconstructing and compensating the three-dimensional coordinates of the calibration points in the left camera coordinate system to obtain compensated three-dimensional coordinates; converting the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration points in the ideal coordinate system; constructing an error function according to the difference between the actual ideal coordinates and the reconstructed ideal coordinates of the calibration points in the ideal coordinate system, and optimizing and solving the error function to obtain the calibration result of the dual cameras.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, specifically to a dual-camera calibration processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Dual-camera calibration is a core step in building a stereo vision system. Its core purpose is to solve the intrinsic parameters (focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (rotation matrix, translation vector) of the two cameras, thereby establishing a precise mapping relationship between the camera pixel coordinate system and the three-dimensional world coordinate system, providing reliable parameter support for subsequent core tasks such as three-dimensional measurement and target localization.

[0003] Currently, the mainstream methods for dual-camera calibration all use planar calibration boards (such as checkerboard calibration boards or circular dot matrix calibration boards). The core principle is as follows: place the planar calibration board in the common field of view of the two cameras, assume that the plane on which the calibration board is located is the Z=0 plane of the three-dimensional world coordinate system, extract the pixel coordinates of the calibration points (feature points) on the calibration board, construct an error function based on the projection relationship, project the three-dimensional feature points onto the camera imaging plane, and solve the intrinsic and extrinsic parameters of the camera through iterative optimization algorithms.

[0004] However, the existing planar calibration plate calibration methods are insufficient to meet the calibration efficiency and accuracy requirements of industrial-grade high-precision calibration. Summary of the Invention

[0005] This application provides a dual-camera calibration processing method, apparatus, electronic device, and storage medium to improve the accuracy of dual-camera calibration.

[0006] In a first aspect, embodiments of this application also provide a dual-camera calibration processing method, including: The system acquires various sets of calibration board images captured by dual cameras during the movement of a planar calibration board along a set direction with fixed step sizes, driven by a high-precision control platform. It also acquires the actual ideal coordinates of the calibration points on the planar calibration board in an ideal coordinate system. Each set of calibration board images includes an image from the left camera and an image from the right camera. The ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane. M represents the total number of sets of calibration board images. Based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration points in each set of calibration board images, the three-dimensional coordinates of the calibration points in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. The compensated three-dimensional coordinates are transformed to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system; An error function is constructed based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates. The error function is then optimized to obtain the calibration results of the dual cameras.

[0007] Secondly, embodiments of this application also provide a dual-camera calibration processing device, comprising: The calibration board image module is used to acquire various sets of calibration board images collected by dual cameras during the movement of the planar calibration board along a set direction with fixed step sizes driven by a high-precision control platform, as well as the actual ideal coordinates of the calibration points on the planar calibration board in the ideal coordinate system; wherein, each set of calibration board images includes a left camera image and a right camera image; the ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane; M is the total number of sets of calibration board images; The 3D reconstruction module is used to reconstruct and compensate for the 3D coordinates of the calibration point in the left camera coordinate system based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration point in each set of calibration board images, so as to obtain the compensated 3D coordinates. The ideal coordinate reconstruction module is used to transform the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system. The error optimization module is used to construct an error function based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates, and to optimize and solve the error function to obtain the calibration results of the dual cameras.

[0008] Thirdly, embodiments of this application also provide an electronic device, which includes: One or more processors; Storage device for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the dual-camera calibration processing methods provided in the embodiments of this application.

[0009] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the dual-camera calibration processing methods provided in embodiments of this application.

[0010] Fifthly, embodiments of this application also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements any of the dual-camera calibration processing methods provided in embodiments of this application. Attached Figure Description

[0011] Figure 1This is a schematic flowchart of a dual-camera calibration processing method provided according to an embodiment of this application; Figure 2 This is a flowchart illustrating another dual-camera calibration processing method provided according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a dual-camera calibration processing device according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the dual-camera calibration processing method of the embodiments of this application. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0013] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] Figure 1 This is a path diagram of a dual-camera calibration processing method provided according to an embodiment of this application. This embodiment is applicable to high-precision stereo vision measurement, 3D reconstruction, etc., and can be executed by a dual-camera calibration processing device. This dual-camera calibration processing device can be implemented in hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes: S101. Acquire each set of calibration board images collected by the dual cameras during the process of the planar calibration board moving along a set direction with a fixed step size on a high-precision control platform, as well as the actual ideal coordinates of the calibration points on the planar calibration board in the ideal coordinate system; wherein, each set of calibration board images includes a left camera image and a right camera image; the ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane; M is the total number of sets of calibration board images; S102. Based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration points in each set of calibration board images, the three-dimensional coordinates of the calibration points in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. S103. Transform the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system; S104. Construct an error function based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates, and optimize the error function to obtain the calibration results of the dual cameras.

[0015] In this embodiment, a two-dimensional planar calibration plate is placed on a high-precision mobile control console. The console moves the calibration plate along a preset direction (parallel to the camera's imaging plane or perpendicular to the camera's optical axis) at fixed distances (e.g., 1 cm). After each movement, the dual cameras acquire a set of calibration plate images, i.e., from near depth of field to far depth of field, acquiring one set every 1 cm, for a total of M (e.g., 15) sets of calibration images. Each set of calibration images includes one image from the left camera and one image from the right camera.

[0016] The actual physical coordinates of the centers (i.e., calibration points) of all circles on the plane calibration plate are known. The one-dimensional coordinates of the plane calibration plate at the corresponding position in the M / 2 (e.g., the 8th group) image are set to Z=0. The Z coordinates of other positions are determined by the step distance of the high-precision moving control console, thus obtaining the actual ideal coordinates of each calibration point in each group of calibration plate images in the ideal coordinate system. ).

[0017] The calibration parameters for the dual cameras include: intrinsic parameters of the left camera (focal length and principal point coordinates), intrinsic parameters of the right camera (focal length and principal point coordinates), the transformation matrix from the right camera to the left camera (rotation matrix and translation vector), and camera distortion parameters. For example, using the intrinsic parameters of the left camera, the intrinsic parameters of the right camera, and the transformation matrix from the right camera to the left camera, the original 3D coordinates of the calibration point in the left camera coordinate system are reconstructed; using the distortion parameters of the left camera, distortion compensation is performed on the original 3D coordinates of the calibration point to obtain the compensated 3D coordinates.

[0018] The compensated 3D coordinates are transformed to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system. The actual ideal coordinates are then calculated. The difference between the reconstructed ideal coordinates of the calibration points and the target camera is used to construct the following error function. The calibration results of the dual cameras are obtained by iteratively solving the error function. The calibration results of the dual cameras include the calibration values ​​of at least one of the following parameters to be calibrated: the intrinsic parameters of the left camera (focal length and principal point coordinates), the intrinsic parameters of the right camera (focal length and principal point coordinates), the transformation matrix from the right camera to the left camera (rotation matrix and translation vector), and the camera distortion parameters.

[0019] In the technical solution of this application, the calibration parameters of dual cameras are used. Based on the pixel coordinates in each group of calibration board images, the calibration points are reconstructed in three dimensions and distortion is compensated in the coordinate system of the left camera to obtain the compensated three-dimensional coordinates. The compensated three-dimensional coordinates are transformed to the ideal coordinate system to obtain the reconstructed ideal coordinates. The actual ideal coordinates of each calibration point in the ideal coordinate system are used as the reference to calculate and minimize the error of the reconstructed ideal coordinates of the calibration points, so that the error is reduced to a low level, thereby significantly improving the calibration accuracy.

[0020] Compared to traditional camera calibration methods that minimize projection errors from 2D images, this approach offers the advantage of directly performing coordinate transformation and error optimization in 3D space. It is independent of specific camera models, better aligns with actual 3D measurement needs, and improves 3D measurement accuracy. Furthermore, by controlling the movement of the planar calibration plate on a high-precision mobile control console, a set of calibration plate images is acquired after each movement. This allows for the construction of 3D points using the 2D pixel coordinates in the calibration plate images corresponding to the Z=0 plane, and also leverages the Z-plane intervals corresponding to each set of calibration plate images after each movement (e.g., acquiring one set every 1cm, for a total of 15 sets, corresponding to the Z-plane interval [-7,7]). Based on this rich 3D spatial information, calibration accuracy is further enhanced. It should be noted that initial values ​​for the parameters to be calibrated are provided using known structural parameters, and the error function is optimized based on these initial values, further improving the efficiency of the optimization solution.

[0021] The technical solution of this embodiment minimizes errors in three-dimensional space by using the actual ideal coordinates of each calibration point in the ideal coordinate system as a reference, thus achieving dual-camera calibration in three-dimensional space, which is more in line with the needs of three-dimensional measurement. By moving the calibration plate to collect multiple sets of images covering the depth of field, rich stereoscopic spatial information is introduced, further improving the calibration accuracy.

[0022] Figure 2 This is a flowchart illustrating another dual-camera calibration processing method provided in an embodiment of this application. The technical solution of this embodiment is further refined based on the above-described technical solution. See also... Figure 2The dual-camera calibration processing method shown includes: S201. Acquire each set of calibration board images collected by the dual cameras during the process of the planar calibration board moving along a set direction with a fixed step size on a high-precision control platform, as well as the actual ideal coordinates of the calibration points on the planar calibration board in the ideal coordinate system; wherein, each set of calibration board images includes a left camera image and a right camera image; the ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane; M is the total number of sets of calibration board images; S202. Based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration points in each set of calibration board images, the three-dimensional coordinates of the calibration points in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. In one optional implementation, based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration point in each set of calibration board images, the three-dimensional coordinates of the calibration point in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. This includes: using the calibration parameters of the dual cameras, reconstructing the original three-dimensional coordinates of the calibration point in the left camera coordinate system based on the pixel coordinates of the calibration point in each set of calibration board images; and using a polynomial to perform distortion compensation on the original three-dimensional coordinates in the left camera coordinate system to obtain the compensated three-dimensional coordinates. Among them, (X) Y Z) represents the original three-dimensional coordinates, ( () represents the compensated three-dimensional coordinates. () represents the left pixel coordinates. and denoted as the coefficients of the polynomial to be solved.

[0023] In this embodiment, the intrinsic parameters of the left camera, the intrinsic parameters of the right camera, and the transformation matrix from the right camera to the left camera are used to reconstruct the original 3D coordinates of the calibration point in the left camera coordinate system; a polynomial is used to compensate for the distortion of the original 3D coordinates in the left camera coordinate system to obtain the compensated 3D coordinates. Among them, (X) Y Z) represents the original three-dimensional coordinates, ( () represents the compensated three-dimensional coordinates. () represents the left pixel coordinates. and denoted as the coefficients of the polynomial to be solved.

[0024] To address the issue of lens distortion, related technologies employ camera models for distortion compensation. However, these methods compensate based on the pixel coordinates of calibration points, which may result in only accurate pixel coordinates (projection) after compensation, while the 3D coordinates remain inaccurate. In contrast, the embodiments of this application employ a polynomial to directly compensate for the distortion of the original 3D coordinates of the calibration points in the left camera coordinate system. This effectively improves the accuracy of the compensated 3D coordinates, thereby enhancing 3D measurement accuracy in high-precision stereo vision measurement, 3D reconstruction, and other scenarios, without relying on a specific camera projection model.

[0025] In one optional implementation, the step of using the calibration parameters of the dual cameras to reconstruct the original three-dimensional coordinates of the calibration point in the left camera coordinate system based on the pixel coordinates of the calibration point in each set of calibration board images includes: normalizing the left pixel coordinates of the calibration point in the left calibration board image using the focal length and principal point coordinates in the intrinsic parameters of the left camera to obtain the normalized left pixel coordinates. in,( )and( The left pixel coordinates before and after normalization are shown below. and The focal length is the intrinsic parameter of the left camera. and The coordinates of the principal point in the intrinsic parameters of the left camera are used; the right pixel coordinates of the calibration point in the right calibration board image are normalized using the focal length and principal point coordinates in the intrinsic parameters of the right camera, resulting in the normalized right pixel coordinates: in, and These are the right pixel coordinates before and after normalization, respectively. and The focal length is the intrinsic parameter of the right camera. and The coordinates of the principal point in the intrinsic parameters of the right camera are used; the original 3D coordinates of the calibration point in the coordinate system of the left camera are reconstructed using the rotation matrix and translation vector from the right camera to the left camera. in, , ; and Let be the rotation matrix and translation vector from the right camera to the left camera.

[0026] To eliminate the influence of camera intrinsic parameters, pixel coordinates are transformed into unit direction vectors emitted from the camera's optical center to the planar calibration plate. Specifically, the principal point coordinates are subtracted from the pixel coordinates to move the origin of the coordinate system from the upper left corner of the image to the intersection of the optical axis and the imaging plane; furthermore, the coordinates are divided by the focal length to eliminate the image scaling effect caused by different camera focal lengths, thus unifying the pixel coordinates of the left and right images into the same metric space.

[0027] Based on this, the depth of the calibration point is calculated using epipolar geometric constraints, and the calculation formula is as follows: After obtaining the depth Z, the X and Y coordinates of the calibration point are directly calculated based on the ray direction of the left camera to obtain the original 3D coordinates. Normalization eliminates the influence of differences in camera intrinsic parameters. The depth solution method based on epipolar geometry constraints can efficiently and accurately determine the original 3D coordinates of the calibration point.

[0028] S203. Transform the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system; S204. Construct an error function based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates, and optimize the error function to obtain the calibration results of the dual cameras; For example, calculate the actual ideal coordinates ( The difference between the reconstructed ideal coordinates of the calibration points and the original coordinates is used to construct the following error function: ; in, Let N be the error function between the actual ideal coordinates and the reconstructed ideal coordinates, and let N be the number of calibration points. To reconstruct the ideal coordinates, R and T are the rotation matrix and translation vector between the left camera coordinate system and the ideal coordinate system, respectively. The calibration results of the two cameras can be obtained by iteratively solving the error function (e.g., using the Levenberg-Marquardt algorithm).

[0029] S205. Based on the calibration results of the dual cameras, determine the optimized reconstructed ideal coordinates of the calibration points in the ideal coordinate system; S206. Based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the optimized reconstructed ideal coordinates, construct an error table for the calibration point; the error table is used to compensate for the error in the reconstructed three-dimensional coordinates of each point in the object to be measured during the object measurement process.

[0030] For each calibration point, the reconstructed ideal coordinates corresponding to the calibration result after optimization are obtained. The difference between the reconstructed ideal coordinates and the actual ideal coordinates of the calibration point is calculated as the 3D spatial error of that calibration point, and an error table is constructed based on the 3D spatial errors of all calibration points. The error table includes the 3D coordinates of each calibration point and its corresponding 3D spatial error. Even if the residuals of the 3D points are reduced to a small level through iterative optimization, there will still be a small residual residual. To address this issue, this embodiment of the application constructs an error table for each calibration point, effectively compensating for this residual residual during the actual object measurement process, thereby further improving the accuracy of object measurement.

[0031] The technical solution of this embodiment abandons the traditional camera model that relies on pixel coordinates for distortion compensation and uses a polynomial to directly compensate for the distortion of the original three-dimensional coordinates in the left camera coordinate system, which effectively improves the accuracy of the compensated three-dimensional coordinates and does not depend on a specific camera projection model; and by constructing an error table including the three-dimensional coordinates of each calibration point and their three-dimensional spatial errors, it is used to compensate for the residual errors that still exist after optimization in actual three-dimensional measurement, which can further improve the accuracy of three-dimensional measurement.

[0032] In one optional embodiment, the method further includes: acquiring dual-object images obtained by dual-camera image acquisition of the object under test; reconstructing and distortion compensation of the three-dimensional coordinates of the target point in the left camera coordinate system based on the calibrated parameters of the dual cameras and the pixel coordinates of the target point on the object under test in the dual-object images, to obtain the compensated three-dimensional coordinates; and using the error table to perform secondary error compensation on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point.

[0033] In the object measurement phase, dual-image datasets of the object under test are acquired using two cameras. The pixel coordinates (left and right pixel coordinates) of the target point on the object are extracted from these dual-image datasets. The left pixel coordinates of the target point are normalized using the calibrated intrinsic parameters of the left camera (focal length and principal point coordinates). Similarly, the right pixel coordinates of the target point are normalized using the calibrated intrinsic parameters of the right camera (focal length and principal point coordinates). A rotation matrix from the calibrated right camera to the left camera is then used to obtain the normalized right pixel coordinates. Translation vector The original 3D coordinates of the target point in the left camera coordinate system are reconstructed using calibrated polynomial coefficients. and The original 3D coordinates of the target point are distorted to obtain the compensated 3D coordinates. The reference point and its error are determined from the error table, and the error of the reference point is used to perform secondary error compensation on the compensated 3D coordinates of the target point to obtain the final 3D coordinates. Through this dual correction mechanism of polynomial distortion compensation and secondary compensation from the error table, the minor errors remaining in the calibration stage are effectively eliminated, further improving the accuracy of the 3D strategy.

[0034] In one optional implementation, the step of using the error table to perform secondary error compensation on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point includes: determining the cube space to which the target point belongs from the error table; wherein each vertex of the cube space is a calibration point; calculating the error value of the target point based on the error value corresponding to the calibration point at each vertex of the cube space; and using the error value of the target point to perform secondary error compensation on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point.

[0035] For example, based on the node division method of the error table, the cubic space to which the target point belongs is determined from the error table; the eight vertices of this cubic space are all calibration points in the calibration stage. The error values ​​corresponding to the calibration points at each vertex are obtained from the error table; using a linear interpolation method, the error value of the target point is calculated based on the three-dimensional coordinates of each vertex in the left camera coordinate system, the compensated three-dimensional coordinates of the target point, and the error values ​​of each vertex; and the error value of the target point is subtracted from the compensated three-dimensional coordinates of the target point to obtain the final three-dimensional coordinates of the target point. By using the errors of each vertex in the target's cubic space to interpolate and calculate the error of the target point, the calculation efficiency of the target point error is improved, thereby improving the efficiency of three-dimensional measurement.

[0036] Figure 3 This is a schematic diagram of a dual-camera calibration processing device according to an embodiment of this application. This embodiment is applicable to high-precision stereo vision measurement, 3D reconstruction, and other similar applications. The dual-camera calibration processing device can be implemented in hardware and / or software, and can be configured in an electronic device. (Reference) Figure 3 The specific structure of the dual-camera calibration processing device 300 is as follows: The calibration board image module 310 is used to acquire each set of calibration board images collected by the dual cameras during the process of the planar calibration board moving along a set direction with a fixed step size driven by the high-precision control platform, as well as the actual ideal coordinates of the calibration points in the planar calibration board in the ideal coordinate system; wherein, each set of calibration board images includes a left camera image and a right camera image; the ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane; M is the total number of sets of calibration board images; The 3D reconstruction module 320 is used to reconstruct and compensate for the 3D coordinates of the calibration point in the left camera coordinate system based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration point in each set of calibration board images, so as to obtain the compensated 3D coordinates. The ideal coordinate reconstruction module 330 is used to transform the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system. The error optimization module 340 is used to construct an error function based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates, and to optimize and solve the error function to obtain the calibration results of the dual cameras.

[0037] In one alternative implementation, the three-dimensional reconstruction module 320 includes: The original coordinate reconstruction unit is used to reconstruct the original three-dimensional coordinates of the calibration point in the left camera coordinate system based on the pixel coordinates of the calibration point in each set of calibration board images using the calibration parameters of the dual cameras. The distortion compensation unit is used to perform distortion compensation on the original three-dimensional coordinates in the left camera coordinate system using a polynomial, to obtain the compensated three-dimensional coordinates: ; ; ; Among them, (X) Y Z) represents the original three-dimensional coordinates, ( () represents the compensated three-dimensional coordinates. () represents the left pixel coordinates. and denoted as the coefficients of the polynomial to be solved.

[0038] In one optional implementation, the original coordinate reconstruction unit is specifically used for: Using the focal length and principal point coordinates from the left camera's intrinsic parameters, the left pixel coordinates of the calibration point in the left calibration board image are normalized to obtain the normalized left pixel coordinates: in,( )and( The left pixel coordinates before and after normalization are shown below. and The focal length is the intrinsic parameter of the left camera. and The coordinates of the principal point in the intrinsic parameters of the left camera; Using the focal length and principal point coordinates from the intrinsic parameters of the right camera, the right pixel coordinates of the calibration point in the right calibration board image are normalized to obtain the normalized right pixel coordinates: in, and These are the right pixel coordinates before and after normalization, respectively. and The focal length is the intrinsic parameter of the right camera. and The coordinates of the principal point in the intrinsic parameters of the right camera; Using the rotation matrix and translation vector from the right camera to the left camera, the original 3D coordinates of the calibration point in the left camera coordinate system are reconstructed: in, , ; and Let be the rotation matrix and translation vector from the right camera to the left camera.

[0039] In one optional embodiment, the device 300 further includes an error table construction module, the error table construction module comprising: The reconstructed ideal coordinate unit is used to determine the optimized reconstructed ideal coordinates of the calibration point in the ideal coordinate system based on the calibration results of the dual cameras. The error table construction unit is used to construct an error table for the calibration point based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the optimized reconstructed ideal coordinates; the error table is used to compensate for errors in the reconstructed three-dimensional coordinates of each point in the object to be measured during the object measurement process.

[0040] In one optional embodiment, the device 300 further includes an object measurement module, the object measurement module comprising: The object image acquisition unit is used to acquire dual-object images obtained by dual cameras acquiring images of the object under test. The object coordinate compensation unit is used to reconstruct and compensate for the three-dimensional coordinates of the target point in the left camera coordinate system based on the calibrated parameters of the dual cameras and the pixel coordinates of the target point on the object to be measured in the dual object image, so as to obtain the compensated three-dimensional coordinates. The secondary compensation unit is used to perform secondary error compensation on the compensated three-dimensional coordinates using the error table to obtain the final three-dimensional coordinates of the target point.

[0041] In one optional implementation, the secondary compensation unit is specifically used for: The cube space to which the target point belongs is determined from the error table; wherein each vertex of the cube space is a calibration point; Calculate the error value of the target point based on the error values ​​corresponding to the calibration points at each vertex of the cube space; Using the error value of the target point, a second error compensation is performed on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point.

[0042] The dual-camera calibration processing device provided in this application embodiment can execute the dual-camera calibration processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the dual-camera calibration processing method.

[0043] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0044] Figure 4 This is a schematic diagram of the structure of an electronic device 410 implementing the dual-camera calibration processing method of the embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0045] like Figure 4 As shown, the electronic device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the electronic device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0046] Multiple components in electronic device 410 are connected to I / O interface 415, including: input unit 416, such as keyboard, mouse, etc.; output unit 417, such as various types of displays, speakers, etc.; storage unit 418, such as disk, optical disk, etc.; and communication unit 419, such as network card, modem, wireless transceiver, etc. Communication unit 419 allows electronic device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0047] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as the dual-camera calibration processing method.

[0048] In some embodiments, the dual-camera calibration processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the dual-camera calibration processing method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured as the dual-camera calibration processing method by any other suitable means (e.g., by means of firmware).

[0049] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0050] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device, such that when executed by the processor, the computer programs cause the functions / operations specified in the path diagrams and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0051] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0052] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0053] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0054] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0055] It should be understood that the various paths shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0056] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A dual-camera calibration processing method, characterized in that, include: The system acquires various sets of calibration board images captured by dual cameras during the movement of a planar calibration board along a set direction with fixed step sizes, driven by a high-precision control platform. It also acquires the actual ideal coordinates of the calibration points on the planar calibration board in an ideal coordinate system. Each set of calibration board images includes an image from the left camera and an image from the right camera. The ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane. M represents the total number of sets of calibration board images. Based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration points in each set of calibration board images, the three-dimensional coordinates of the calibration points in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. The compensated three-dimensional coordinates are transformed to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system; An error function is constructed based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates. The error function is then optimized to obtain the calibration results of the dual cameras.

2. The method according to claim 1, characterized in that, The step involves reconstructing and distortion-compensating the 3D coordinates of the calibration points in the left camera coordinate system based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration points in each set of calibration board images, to obtain the compensated 3D coordinates, including: Using the calibration parameters of dual cameras, the original three-dimensional coordinates of the calibration points in the left camera coordinate system are reconstructed based on the pixel coordinates of the calibration points in each set of calibration board images. The original 3D coordinates in the left camera coordinate system are distorted using a polynomial to obtain the compensated 3D coordinates: ； ； ； Among them, (X) Y Z) represents the original three-dimensional coordinates, ( () represents the compensated three-dimensional coordinates. () represents the left pixel coordinates. and denoted as the coefficients of the polynomial to be solved.

3. The method according to claim 2, characterized in that, The calibration parameters using dual cameras, based on the pixel coordinates of the calibration points in each set of calibration board images, reconstruct the original three-dimensional coordinates of the calibration points in the left camera coordinate system, including: Using the focal length and principal point coordinates from the left camera's intrinsic parameters, the left pixel coordinates of the calibration point in the left calibration board image are normalized to obtain the normalized left pixel coordinates: in,( )and( The left pixel coordinates before and after normalization are shown below. and The focal length is the intrinsic parameter of the left camera. and The coordinates of the principal point in the intrinsic parameters of the left camera; Using the focal length and principal point coordinates from the intrinsic parameters of the right camera, the right pixel coordinates of the calibration point in the right calibration board image are normalized to obtain the normalized right pixel coordinates: in, and These are the right pixel coordinates before and after normalization, respectively. and The focal length is the intrinsic parameter of the right camera. and The coordinates of the principal point in the intrinsic parameters of the right camera; Using the rotation matrix and translation vector from the right camera to the left camera, the original 3D coordinates of the calibration point in the left camera coordinate system are reconstructed: in, , ; and Let be the rotation matrix and translation vector from the right camera to the left camera.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Based on the calibration results of the dual cameras, the optimized reconstructed ideal coordinates of the calibration points in the ideal coordinate system are determined; Based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the optimized reconstructed ideal coordinates, an error table for the calibration point is constructed; the error table is used to compensate for the error in the reconstructed three-dimensional coordinates of each point in the object to be measured during the object measurement process.

5. The method according to claim 4, characterized in that, The method further includes: Acquire dual-object images obtained by dual-camera image acquisition of the object under test; Based on the calibrated parameters of the dual cameras and the pixel coordinates of the target point on the object under test in the dual object image, the three-dimensional coordinates of the target point in the left camera coordinate system are reconstructed and distortion compensated to obtain the compensated three-dimensional coordinates. Using the aforementioned error table, a second error compensation is performed on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point.

6. The method according to claim 5, characterized in that, The step of using the error table to perform secondary error compensation on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point includes: The cube space to which the target point belongs is determined from the error table; wherein each vertex of the cube space is a calibration point; Calculate the error value of the target point based on the error values ​​corresponding to the calibration points at each vertex of the cube space; Using the error value of the target point, a second error compensation is performed on the compensated three-dimensional coordinates to obtain the final three-dimensional coordinates of the target point.

7. A dual-camera calibration processing device, characterized in that, include: The calibration board image module is used to acquire various sets of calibration board images collected by dual cameras during the movement of the planar calibration board along a set direction with fixed step sizes driven by a high-precision control platform, as well as the actual ideal coordinates of the calibration points on the planar calibration board in the ideal coordinate system; wherein, each set of calibration board images includes a left camera image and a right camera image; the ideal coordinate system is a three-dimensional coordinate system with the plane of the planar calibration board corresponding to the M / 2th set of images as the Z=0 plane; M is the total number of sets of calibration board images; The 3D reconstruction module is used to reconstruct and compensate for the 3D coordinates of the calibration point in the left camera coordinate system based on the calibration parameters of the dual cameras and the pixel coordinates of the calibration point in each set of calibration board images, so as to obtain the compensated 3D coordinates. The ideal coordinate reconstruction module is used to transform the compensated three-dimensional coordinates to the ideal coordinate system to obtain the reconstructed ideal coordinates of the calibration point in the ideal coordinate system. The error optimization module is used to construct an error function based on the difference between the actual ideal coordinates of the calibration point in the ideal coordinate system and the reconstructed ideal coordinates, and to optimize and solve the error function to obtain the calibration results of the dual cameras.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the dual-camera calibration processing method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the dual-camera calibration processing method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the dual-camera calibration processing method according to any one of claims 1-6.

Images