A projector-assisted binocular vision system calibration method and related apparatus

Abstract

The application discloses a kind of projector assisted binocular vision system calibration method, binocular vision system calibration device, binocular vision system calibration equipment, computer readable storage medium, method includes: based on Zhang calibration method calculated internal and external parameters respectively carry out calibration space calculation, obtain calibration space, and based on the size of calibration board and each described calibration space calculation is used to guide the target frame of calibration board placement;When the calibration board placement is completed, control projector projects preset image on the calibration board, control camera gathers calibration board image;Based on the re-projection error and phase consistency constraint error determined by the calibration board image is carried out joint optimization, obtains double target calibration error;Based on left monocular system calibration error, right monocular system calibration error, the double target calibration error, monocular and binocular parameter consistency error, three-dimensional point cloud consistency error is carried out joint optimization, obtains target calibration error;Based on the target calibration error carries out calibration processing.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and more specifically, relates to a projector-assisted binocular vision system calibration method, a binocular vision system calibration device, a binocular vision system calibration equipment, and a computer-readable storage medium. Background Technology

[0002] Structured light measurement is based on actively projecting a specific light pattern, capturing the deformed pattern modulated on the surface of an object, and then calculating the object's three-dimensional coordinates using triangulation. Compared to traditional contact measurement methods, structured light measurement technology has the advantages of high efficiency, simplicity, and the ability to acquire the three-dimensional information of the measured object non-contactly, and has been widely used in fields such as industrial measurement.

[0003] In related technologies, for the measurement of large-area workpieces, traditional binocular structured light systems struggle to balance accuracy and measurement range due to factors such as low calibration accuracy, insufficient calibration range, and similar calibration data. This results in high measurement accuracy in the central area but low accuracy in the edge areas, leading to a small usable high-precision area. Furthermore, the reconstructed area is typically the intersection of the common field of view of the binocular camera and the structured light projection area, further reducing the size of the reconstructed area and severely impacting the measurement efficiency of large-area workpieces.

[0004] Therefore, how to expand the high-precision area of ​​the binocular vision system after calibration and avoid affecting the measurement efficiency of large surface workpieces is a key issue of concern to those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a projector-assisted binocular vision system calibration method, a binocular vision system calibration device, a binocular vision system calibration equipment, and a computer-readable storage medium, so as to expand the high-precision area of ​​the binocular vision system after calibration and improve the measurement efficiency of large surface workpieces.

[0006] To address the above-mentioned deficiencies or improvement needs of existing technologies, this invention provides a projector-assisted binocular vision system calibration method, comprising:

[0007] Based on Zhang's calibration method, the monocular and binocular structured light systems were pre-calibrated to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector;

[0008] Based on the endo- and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, calibration spaces are calculated to obtain the calibration spaces of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. A target frame is then calculated based on the dimensions of the calibration board and each calibration space. The target frame is used to guide the placement of the calibration board via a projector.

[0009] After the calibration board is placed, the projector is controlled to project a preset image onto the calibration board, and the camera is controlled to capture the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image;

[0010] The reprojection error and phase consistency constraint error determined by the calibration board image are jointly optimized to obtain the dual-target calibration error;

[0011] The target calibration error is obtained by jointly optimizing the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud.

[0012] Calibration processing is performed based on the target calibration error.

[0013] Optionally, the monocular and binocular structured light systems are pre-calibrated based on Zhang's calibration method to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, including:

[0014] Both the left and right cameras acquire pre-calibrated images;

[0015] Based on the acquired pre-calibrated images, Zhang's calibration process is performed to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system.

[0016] Optionally, calibration space calculations are performed based on the endo- and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. A target bounding box is then calculated based on the size of the calibration board and each calibration space, including:

[0017] Based on the intrinsic and extrinsic parameters of the left monocular structured light system, the extrinsic and extrinsic parameters of the right monocular structured light system, and the extrinsic and extrinsic parameters of the binocular structured light system, calibration space calculations are performed to obtain the calibration space of the left monocular structured light system, the calibration space of the right monocular structured light system, and the calibration space of the binocular structured light system.

[0018] Based on the size of the calibration plate, each calibration space is divided into regions to obtain the corresponding calibration plate placement pose;

[0019] Based on the placement pose of each calibration board, the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system are calculated;

[0020] The target bounding box is obtained by connecting the pixel coordinates of the four vertices of the calibration board.

[0021] Optionally, the calibration plate is guided to be placed using a projector, including:

[0022] The projector projects the target frame, and the camera image displays the target frame as an aid to guide the placement of the calibration plate.

[0023] Optionally, the reprojection error and phase consistency constraint error determined based on the calibration board image are jointly optimized to obtain the bi-target calibration error, including:

[0024] The corner points of the calibration board are extracted based on the full-brightness image of the calibration board image, and the reprojection error is obtained by constructing the error of the corner points of the calibration board using Zhang's calibration method.

[0025] The phase-shifting method is used to perform phase expansion on the images of the calibration board, excluding the fully bright images, to obtain the horizontal absolute phase and the vertical absolute phase.

[0026] Errors are constructed based on the phase differences between corresponding corner points in the calibration plate, the lateral absolute phase, and the longitudinal absolute phase to obtain phase consistency constraint errors;

[0027] The bi-target positioning error is obtained by minimizing the first preset weight, the reprojection error, and the phase consistency constraint error using the Levenberg-Marquardt algorithm.

[0028] Optionally, the target calibration error is obtained by joint optimization based on the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the binocular system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud, including:

[0029] Error construction processing is performed based on the corresponding monocular reprojection error and the corresponding monocular phase error to obtain the calibration error of the left monocular system and the calibration error of the right monocular system;

[0030] Error construction processing is performed based on the intrinsic and extrinsic parameters of the monocular structured light system and the intrinsic and extrinsic parameters of the binocular structured light system to obtain the monocular and binocular parameter consistency error.

[0031] The three-dimensional point coordinates of the binocular overlapping area between the monocular structured light system and the binocular structured light system are made consistent, and error construction processing is performed to obtain the three-dimensional point cloud consistency error.

[0032] The target calibration error is obtained by minimizing the second preset weight, the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the binocular system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud using the Levenberg-Marquardt algorithm.

[0033] This application also provides a projector-assisted binocular vision system calibration device, comprising:

[0034] The pre-calibration module is used to pre-calibrate the monocular and binocular structured light systems based on Zhang's calibration method, and to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector.

[0035] The calibration board placement module is used to calculate the calibration space based on the endo- and endo-parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. It then calculates a target frame based on the size of the calibration board and each calibration space. The target frame is used to guide the placement of the calibration board via a projector.

[0036] The image acquisition module is used to control the projector to project a preset image onto the calibration board after the calibration board is placed, and to control the camera to acquire the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image;

[0037] The monocular and binocular joint optimization module is used to perform joint optimization based on the reprojection error and phase consistency constraint error determined by the calibration board image to obtain the binocular calibration error;

[0038] The target optimization module is used to perform joint optimization based on the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud to obtain the target calibration error;

[0039] The calibration processing module is used to perform calibration processing based on the target calibration error.

[0040] Optionally, the calibration board placement module is specifically used to perform calibration space calculations based on the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; to divide each calibration space into regions based on the size of the calibration board, to obtain the corresponding calibration board placement pose; to calculate the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system based on each calibration board placement pose; and to connect the pixel coordinates of the four vertices of the calibration board to obtain the target bounding box.

[0041] This application also provides a binocular vision system calibration device, including:

[0042] Memory, used to store computer programs;

[0043] A processor is configured to implement the steps of the binocular vision system calibration method as described above when executing the computer program.

[0044] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the binocular vision system calibration method described above.

[0045] This application provides a projector-assisted binocular vision system calibration method, comprising: pre-calibrating a monocular and binocular structured light system based on Zhang's calibration method to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector; calculating the calibration space based on the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system respectively to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, and calculating the calibration space based on the size of the calibration board and each The calibration space calculates the target bounding box; wherein, the target bounding box is used to guide the placement of the calibration board via a projector; after the calibration board is placed, the projector is controlled to project a preset image onto the calibration board, and the camera is controlled to acquire the calibration board image; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image; based on the reprojection error and phase consistency constraint error determined by the calibration board image, joint optimization is performed to obtain the bi-target calibration error; based on the left monocular system calibration error, the right monocular system calibration error, the bi-target calibration error, the mono- and binocular parameter consistency error, and the 3D point cloud consistency error, joint optimization is performed to obtain the target calibration error; calibration processing is performed based on the target calibration error.

[0046] Initial parameters are obtained through pre-calibration based on Zhang's calibration method. Uniform coverage of the calibration board is achieved by combining calibration space calculation and target box guidance. Reprojection and phase errors are jointly optimized using full-brightness and coded images. Furthermore, single- and binocular multi-error analysis is performed for joint optimization, ensuring consistent calibration accuracy across the entire field of view. This method expands the high-precision area, solves the problems of low edge accuracy and small reconstruction area, significantly improves the measurement efficiency of large-area workpieces, and provides high-precision basic data for industrial measurement. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 A flowchart illustrating a projector-assisted binocular vision system calibration method provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of a monocular and binocular structured light system provided in an embodiment of this application;

[0050] Figure 3 This is a flowchart of an initial pre-calibration process provided in an embodiment of this application;

[0051] Figure 4 This is a flowchart of the structured light system guidance and calibration provided in an embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the structured light system guidance and calibration provided in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram illustrating the optimization of dual-target positioning accuracy using phase information, provided in an embodiment of this application.

[0054] Figure 7 This is a flowchart illustrating the optimization of dual-target accuracy using phase information, provided in an embodiment of this application.

[0055] Figure 8 This is a flowchart of the monocular and binocular joint optimization framework provided in the embodiments of this application;

[0056] Figure 9 A schematic diagram of the structure of a projector-assisted binocular vision system calibration device provided in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram of the structure of a binocular vision system calibration device provided in an embodiment of this application. Detailed Implementation

[0058] The purpose of this application is to provide a projector-assisted binocular vision system calibration method, a binocular vision system calibration device, a binocular vision system calibration equipment, and a computer-readable storage medium, so as to expand the high-precision area of ​​the binocular vision system after calibration and improve the measurement efficiency of large surface workpieces.

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0060] The following embodiment illustrates a projector-assisted binocular vision system calibration method provided in this application.

[0061] Please refer to Figure 1 , Figure 1 This is a flowchart of a projector-assisted binocular vision system calibration method provided in an embodiment of this application.

[0062] In this embodiment, the method may include:

[0063] S101, based on Zhang's calibration method, pre-calibrate the monocular and binocular structured light systems to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the intrinsic and extrinsic parameters of the right monocular structured light system, and the intrinsic and extrinsic parameters of the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector.

[0064] In this step, Zhang's calibration method is a classic camera calibration method. By capturing images of the calibration board in different poses, the camera's intrinsic and extrinsic parameters can be solved. In this invention, the left monocular structured light system (left camera + projector), the right monocular structured light system (right camera + projector), and the binocular structured light system (left camera + right camera + projector) are first pre-calibrated to obtain initial extrinsic and extrinsic parameters, providing basic data for subsequent precise calibration. This is because initial parameters are a prerequisite for subsequent calibration space calculations and error optimization; only by first determining the preliminary system model can more refined adjustments be made.

[0065] In this step, the Zhang calibration method is used for pre-calibration, which provides initial parameters for the entire calibration process and establishes preliminary models of monocular and binocular systems. This provides basic data for subsequent calibration space division, target box calculation, and error optimization, ensuring that subsequent steps can be accurately adjusted and optimized based on this data.

[0066] Furthermore, this step may include:

[0067] Step 1: Both the left and right cameras acquire pre-calibrated images;

[0068] Step 2: Perform Zhang's calibration processing based on the acquired pre-calibrated images to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system.

[0069] S102, calibration spaces are calculated based on the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration spaces of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. The target bounding box is then calculated based on the dimensions of the calibration board and each calibration space. The target bounding box is used to guide the placement of the calibration board via a projector.

[0070] Based on S101, the calibration spaces (i.e., the field of view of each system) of the left monocular, right monocular, and binocular structured light systems are calculated. Taking into account the calibration board size, each calibration space is divided into regions, and the placement pose (including position and orientation) of the calibration board is planned to ensure that the calibration board uniformly covers the entire calibration space. Then, the coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system are calculated for each pose, and these coordinates are connected to form the target bounding box. The principle is that by reasonably dividing the region and planning the pose, the calibration data is ensured to be evenly distributed throughout the measurement space, avoiding the problem of insufficient data in edge areas in traditional methods; the target bounding box acts as a visual guidance tool, helping the operator to accurately place the calibration board and ensuring that the calibration board is in the preset optimal pose.

[0071] As can be seen, this step, through calibration space calculation and target box guidance, ensures that the calibration board can be accurately placed according to the planned pose, so that the calibration data can evenly cover the entire measurement space, avoiding the problem of calibration data being concentrated in the central area while missing in the edge area, and laying the foundation for improving the consistency of calibration accuracy of the entire measurement space in the future.

[0072] Furthermore, this step may include:

[0073] Step 1: Based on the intrinsic and extrinsic parameters of the left monocular structured light system, the extrinsic and extrinsic parameters of the right monocular structured light system, and the extrinsic and extrinsic parameters of the binocular structured light system, the calibration space is calculated to obtain the calibration space of the left monocular structured light system, the calibration space of the right monocular structured light system, and the calibration space of the binocular structured light system.

[0074] Step 2: Divide each calibration space into regions based on the size of the calibration board to obtain the corresponding calibration board placement pose;

[0075] Step 3: Calculate the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system based on the placement pose of each calibration board;

[0076] Step 4: Connect the pixel coordinates of the four vertices of the calibration board to obtain the target bounding box.

[0077] The placement of the calibration board guided by a projector may include: projecting a target frame onto the target frame using a camera image as an aid to guide the placement of the calibration board.

[0078] S103, After the calibration board is placed, control the projector to project a preset image onto the calibration board, and control the camera to capture the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image;

[0079] Building upon S102, once the calibration board is positioned according to the target frame, the projector projects a set of preset images, including one fully illuminated image and 48 coded fringe images (24 horizontal and 24 vertical). The fully illuminated image is used to extract corner points of the calibration board, while the coded fringe images are used to calculate the phase using a phase-shifting method to obtain absolute phase information. The camera simultaneously acquires these images, providing data support for subsequent error calculations. The fully illuminated image is used for corner detection, while the coded fringe images carry spatial location information through phase changes. Combining the two allows for the simultaneous acquisition of feature point coordinates and phase information, providing multi-dimensional data for joint optimization.

[0080] This step, by acquiring full-brightness images and coded stripe images, can simultaneously obtain the corner coordinates and phase information of the calibration board. This provides abundant data for subsequent reprojection error calculation (based on calibration board corners) and phase consistency constraint error calculation (based on phase information), ensuring that the error optimization process can comprehensively utilize multiple types of information and improve calibration accuracy.

[0081] S104, based on the reprojection error and phase consistency constraint error determined by the calibration board image, a joint optimization is performed to obtain the dual-target calibration error;

[0082] Building upon S103, this step utilizes corner points extracted from the fully illuminated image to calculate the reprojection error (i.e., the deviation between the actual corner point pixel coordinates and the model's predicted coordinates) using the Zhang calibration method. Then, for the lateral and longitudinal absolute phases obtained from the coded stripe image, a phase consistency constraint error is constructed using the constraint that the phase values ​​of corresponding corner points on the left and right cameras should be equal. These two errors are then combined and minimized using the Levenberg-Marquardt algorithm to obtain the bi-target calibration error. The principle is that the reprojection error reflects the geometric accuracy of the camera model, while the phase consistency error reflects the phase matching accuracy of the structured light system. Joint optimization of both can simultaneously improve the consistency of camera parameters and structured light parameters, avoiding the one-sidedness of relying solely on geometric or phase information.

[0083] This step, by jointly optimizing reprojection error and phase consistency constraint error, fully utilizes the complementarity of geometric and phase features, significantly improving the calibration accuracy of the binocular structured light system. Especially in the edge region, phase constraints compensate for possible deficiencies in geometric features, making the intrinsic and extrinsic parameters of the binocular camera more accurate.

[0084] Furthermore, this step may include:

[0085] Step 1: Extract the corner points of the calibration board based on the full-brightness image of the calibration board image, and construct the error of the corner points of the calibration board using Zhang's calibration method to obtain the reprojection error;

[0086] Step 2: Perform phase expansion on the calibration plate image (excluding the fully bright image) using the phase shift method to obtain the horizontal absolute phase and the vertical absolute phase.

[0087] Step 3: Based on the phase difference between corresponding corner points in the calibration board corner points, lateral absolute phase, and longitudinal absolute phase, the error is constructed to obtain the phase consistency constraint error;

[0088] Step 4: Minimize the first preset weight, reprojection error, and phase consistency constraint error using the Levenberg-Marquardt algorithm to obtain the bi-objective positioning error.

[0089] S105, based on the joint optimization of the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud, the target calibration error is obtained;

[0090] Building upon S104, this step constructs a joint optimization framework, including calibration errors for the left monocular system (combining monocular reprojection and phase errors), calibration errors for the right monocular system, binocular calibration errors, monocular and binocular parameter consistency constraint errors (ensuring consistent camera / projector parameters between the monocular and binocular systems), and 3D point cloud consistency constraint errors (forcing consistency in 3D point coordinates reconstructed by the monocular and binocular systems in overlapping binocular regions). These errors are weighted and minimized using the Levenberg-Marquardt algorithm to obtain the target calibration error. The principle is that the monocular system operates independently in non-overlapping regions, while the binocular system fuses in overlapping regions. Joint optimization eliminates parameter differences and reconstruction errors between the monocular and binocular systems, ensuring consistent calibration accuracy across the entire measurement space (including independent monocular regions and overlapping binocular regions).

[0091] As can be seen, this step, by jointly optimizing various errors of the monocular and binocular systems, not only improves the calibration accuracy of the monocular system in independent regions, but also ensures the consistency of parameters and reconstruction results between the monocular and binocular systems in overlapping regions, significantly reduces the accuracy difference between different regions, achieves high-precision calibration across the entire field of view, and expands the effective measurement range of the system.

[0092] Furthermore, this step may include:

[0093] Step 1: Based on the corresponding monocular reprojection error and the corresponding monocular phase error, perform error construction processing to obtain the calibration error of the left monocular system and the calibration error of the right monocular system;

[0094] Step 2: Based on the intrinsic and extrinsic parameters of the monocular structured light system and the binocular structured light system, error construction processing is performed to obtain the monocular and binocular parameter consistency error;

[0095] Step 3: Consistently match the 3D point coordinates of the binocular overlapping area between the monocular structured light system and the binocular structured light system, and perform error construction processing to obtain the 3D point cloud consistency error;

[0096] Step 4: The Levenberg-Marquardt algorithm is used to minimize and optimize the second preset weight, the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud to obtain the target calibration error.

[0097] S106, Calibration processing is performed based on the target calibration error.

[0098] Building upon S105, this step, following the preceding joint optimization, minimizes the target calibration error. The final calibration process is then performed based on the optimized parameters to obtain accurate intrinsic and extrinsic parameters for both monocular and binocular structured light systems. These parameters accurately describe the geometric relationship between the camera and projector, as well as the imaging model, providing high-precision foundational data for subsequent 3D reconstruction and measurement.

[0099] In summary, this embodiment obtains initial parameters through pre-calibration based on Zhang's calibration method, achieves uniform coverage of the calibration board by combining calibration space calculation and target box guidance, optimizes reprojection and phase errors using full-brightness and coded images, and further integrates single and binocular multi-errors for joint optimization to ensure consistent calibration accuracy across the entire field of view. This method expands the high-precision area, solves the problems of low edge accuracy and small reconstruction area, significantly improves the measurement efficiency of large-area workpieces, and provides high-precision basic data for industrial measurement.

[0100] The following specific embodiment further illustrates the projector-assisted binocular vision system calibration method provided in this application.

[0101] In this embodiment, the method may include:

[0102] S201, build a monocular and binocular structured light vision system, and use Zhang's calibration method to pre-calibrate the intrinsic and extrinsic parameters of the left monocular system, right monocular system and binocular system;

[0103] S202: The calibration spaces of the left and right monocular and binocular systems are independently calculated using the intrinsic and extrinsic parameters of the monocular and binocular systems. The calibration space is then divided into regions based on the calibration board size to plan the placement pose of the calibration board. For the pose of each calibration board, the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system are calculated. The line connecting the corresponding pixel coordinates of the four vertices forms the target box. The projector projects the target box, and the target box is displayed in the camera image of the calibration software to guide the placement of the calibration board.

[0104] S203: After the calibration board is placed in the corresponding pose area, the projector projects a set of images, including a full-brightness image and horizontal and vertical coded stripe images, onto the calibration board, and the camera captures the image of the calibration board.

[0105] S204. In the dual-target calibration calculation, the corner points of the calibration board are extracted from the fully bright image in the acquired image. The Zhang calibration method is used to calculate the reprojection error. The phase is calculated from the coded stripe image to obtain the absolute phase. The joint optimization equation of reprojection error and phase consistency error is constructed by using the constraint that the phase values ​​of the corresponding calibration board corner points in the left and right camera images are equal, thereby optimizing the dual-target calibration accuracy.

[0106] S205 utilizes the more accurate dual-objective calibration results as constraints to construct a joint optimization equation for single and dual-objective calibration, thereby optimizing and improving the consistency between the accuracy of single-objective calibration and the accuracy of single and dual-objective global calibration.

[0107] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a monocular and binocular structured light system provided in an embodiment of this application.

[0108] by Figure 2 The embodiment of the present invention shown is a monocular and binocular structured light system. The monocular and binocular structured light system includes a left camera C. l Projector P j And right camera C r Projector P j The projection range is greater than that of the left camera C. l Field of view and right camera C r Sum of fields of view. Left camera C l With projector P j Composition of left monocular structured light system S l Right camera C r With projector P j Composition of right monocular structured light system S r Left camera C l Right camera C r With projector P j Together they form a binocular structured light system S b .

[0109] In terms of field of view division, region 1 is the left monocular structured light system S. l The field of view, region 5 is the right monocular structured light system S r The field of view, region 3 is the binocular structured light system S b The field of view, which is also the left monocular structured light system S l And right monocular structured light system S r The overlapping areas of the fields of view, region 2 is the left monocular structured light system S lIndependent field of view region, i.e., with the right monocular structured light system S r Non-overlapping fields of view; region 4 is the right monocular structured light system S. r Independent field of view region, i.e., the left monocular structured light system S l Non-overlapping fields of view.

[0110] Furthermore, the calibration method workflow in this embodiment may also include:

[0111] An initial pre-calibration is performed before the formal calibration. Specifically, the calibration plate is placed in area 3, and the projector P... j Each group projects 49 images, where the first image M0 is a fully bright image, and images 2-25 are M1, M2...M... 24 For horizontally coded stripes, images M (26-49) 25 M 26 M 27 ......M 48 Vertically coded stripes.

[0112] Please refer to Figure 3 , Figure 3 This is a flowchart of an initial pre-calibration process provided in an embodiment of this application.

[0113] like Figure 3 For the first three sets of images acquired, the S of the binocular structured light system was calculated using the Zhang calibration method with the full-brightness M0 image. b The calibration parameters are determined using the full brightness M0 and the coded stripe images M1-M. 48 The S of the left monocular structured light system was calculated using an improved method based on Zhang's calibration method. l Calibration parameters and right monocular structured light system S r The calibration parameters include the left camera C. l The intrinsic parameter matrix K l And right camera C r The intrinsic parameter matrix K r Distortion coefficients (k1, k2), extrinsic parameters (rotation matrix R, translation vector T). Measurement range in the depth direction [Z]. min Z max That is, the range of distances from the object to the camera is determined based on the actual measured distance.

[0114] Please refer to Figure 4 , Figure 4 The structured light system guidance and calibration flowchart provided in the embodiments of this application

[0115] like Figure 4 First, the calibration spaces for the left and right monocular and binocular systems are determined. The calibration spaces are then divided into regions, and the placement and pose of the calibration board are planned, as follows:

[0116] Step 1, the method for determining the calibration space is as follows: Left camera C l Field of view 1 is:

[0117]

[0118] in, Left camera C l Horizontal and vertical field of view, The working distance of the camera is Z. work Left camera C l The range of the field of view in the X direction. The working distance of the camera is Z. work Left camera C l The range of the field of view in the Y direction, Z direction work For the measurement range [Z] min Z max The distance value in [ ] represents the distance from the camera's image sensor to the object being measured. Similarly, in the left camera C l In the system coordinate system, the right camera C r The field of view range 5 is:

[0119]

[0120] Where b is the baseline distance. For right camera C r Horizontal and vertical field of view, The working distance of the camera is Z. work Right camera C r The range of the field of view in the X direction. The working distance of the camera is Z. work Right camera C r The range of the field of view in the Y direction, Z direction work For the measurement range [Z] min Z max The distance value is one of the following. Using field of view 1 and field of view 5, the binocular overlap region and monocular non-overlap region can be obtained, where the left monocular non-overlap field of view region 2 is:

[0121]

[0122] Similarly, the non-overlapping field of view region 4 of the right monocular is:

[0123]

[0124] The binocular overlapping field of view region 3 is:

[0125]

[0126] Step 2, the calibration area is divided as follows: First, to avoid gaps between the left monocular non-overlapping field of view region 2, the right monocular non-overlapping field of view region 4, and the binocular overlapping region 3, an expansion coefficient γ is introduced to expand the left monocular non-overlapping field of view region 2 and the right monocular non-overlapping field of view region 4:

[0127]

[0128] Where w is the width of the calibration plate and h is the height of the calibration plate. To ensure that the calibration plates at the beginning and end are aligned with the region boundary, taking the X direction as an example, when the center of each calibration plate moves along the X direction, its left and right edges must satisfy the following:

[0129]

[0130] Among them, X start X is the left endpoint of the region. end N is the right endpoint of the region. x Define the number of grid cells in the X direction. The center point of the first position calibration plate in the X direction, The center point of the calibration plate is located at the end position in the X direction. The distance L between the centers of the calibration plates at the beginning and end positions is also determined. x for:

[0131] L x =X end -X start -w

[0132] The step size Δx and overlap coefficient α in the X direction were calculated. x for:

[0133]

[0134] Similarly, the step size Δy in the Y direction and the overlap coefficient α can be obtained. y for:

[0135]

[0136] Step 3, the attitude planning of the calibration board is as follows: For each calibration area, the calibration board is rotated by an angle θ around the X, Y, and Z axes. x ,θ y ,θ z At the same time, satisfying vertex coordinates P corner Within the calibration space:

[0137] P corner ∈[X min ,X max ]×[Y min ,Y max ]×[Z min Z max ]

[0138] After the calibration plate is rotated, its effective width w eff and effective height h eff for:

[0139]

[0140] Use effective width w eff and effective height h eff Adjust the step size and overlap coefficient as described above, for each set of calibration plate rotation angles (θ) x ,θ y ,θ z Select an appropriate number of grid divisions N. x and N y Make the overlap coefficient α x ,α y ∈(0,1) is a suitable value that meets the requirements of calibration efficiency and calibration accuracy.

[0141] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structured light system guidance and calibration provided in an embodiment of this application.

[0142] like Figure 5 As shown, for the non-overlapping region 2 of the left monocular camera, only the left camera C... l With projector P j For the right monocular non-overlapping region 4, guidance is provided only by the right camera C. r With projector P j For binocular overlap region 3, guided by the left camera C l Right camera C r Projector P j Joint guidance.

[0143] Regarding the placement order of the calibration board poses, starting from one calibration board pose, an optimization algorithm is used to calculate whether the next placement pose of the calibration board is the optimal pose. Specifically, the optimal pose of the calibration board should maximize the accuracy of parameter estimation, which is usually measured by the Fisher Information Matrix (FIM). The larger the determinant of the FIM (D-optimality criterion), the smaller the covariance of the parameter estimation, and the higher the accuracy.

[0144] Specific steps may include:

[0145] Step 1, calculate the current information matrix: for the calibration board pose set {T k}, FIM is:

[0146]

[0147] Among them, J kiIt is the Jacobian matrix of the reprojection error of the i-th point under the k-th pose with respect to the parameters.

[0148] Step 2, evaluate information content: If det(I) exceeds the preset threshold or satisfies the local maximum condition, the current pose is considered sufficiently optimized. Traverse all calibration plate poses planned in the binocular overlap region 3 and select the next optimal pose for placement.

[0149] Furthermore, the four vertices of the calibration board in the optimal pose are calculated at the left camera C. l Projector P j And right camera C r The position in the pixel coordinate system is as follows:

[0150] Step 1: Generate the four vertices P1, P2, P3, P4 of the calibration board in its optimal pose, and calculate their projection onto the projector P through transformation. j Corresponding point P in pixel coordinate system p1 ,P p2 ,P p3 ,P p4 On the left camera C l Corresponding point P in pixel coordinate system cl1 ,P cl2 ,P cl3 ,P cl4 And right camera C r Corresponding point P in pixel coordinate system cr1 ,P cr2 ,P cr3 ,P cr4 Left camera C l The coordinates of a point in the pixel coordinate system are calculated as follows:

[0151] Left camera C l The coordinate system is the coordinate system of the structured light device, with one corner point of the calibration plate at the left camera C. l Three-dimensional coordinates P in coordinate system c (X c ,Y c Z c Taking a point in the camera coordinate system as an example, let's calculate the transformation from 3D points to image coordinates. The relationship between a point in the camera coordinate system and a point (x, y) in the image coordinate system is as follows:

[0152]

[0153] Where f is the focal length of the camera. The relationship between a point (x, y) in the image coordinate system and a point (u, v) in the pixel coordinate system is as follows:

[0154]

[0155] Where dx and dy are the physical dimensions of the pixel in the X and Y directions, respectively, and (u0, v0) is the position of the origin of the image coordinate system in the pixel coordinate system.

[0156] Combining the above equations, we can obtain the three-dimensional point P. c (X c ,Y c Z c (Corresponding to left camera C) l Image pixel coordinates (u l ,v l )for:

[0157]

[0158]

[0159] Step 2, right camera C r The coordinates in the pixel coordinate system are calculated as follows:

[0160] Given that the right camera C is calculated from the first three sets of images r With left camera C l Baseline distance and right camera C r The intrinsic and extrinsic parameter matrices, and the right camera C. r Point P in the coordinate system cr (X cr ,Y cr Z cr (This can be achieved through the right camera C) r The extrinsic parameter matrix from the left camera C l Three-dimensional coordinates P in the coordinate system c (X c ,Y c Z c (Translated from:)

[0161]

[0162] Where R and T are the right camera C r Relative to left camera C l The rotation matrix and translation vector.

[0163] Right camera C r Point P in the coordinate system cr (X cr ,Y cr Z cr Convert to right camera C r Points in the image coordinate system (x r ,y r The relationship is:

[0164]

[0165] Right camera C r Points in the image coordinate system (x r ,y r Convert to a point in pixel coordinates (u) r ,v r The relationship is:

[0166]

[0167] Among them, dx r and dy r Right camera C r The physical dimensions of a pixel in the X and Y directions, (u 0r ,v 0r () is the right camera C r The position of the origin of the image coordinate system in the pixel coordinate system.

[0168] Combining the above equations, we can obtain the three-dimensional point P. cr (X cr ,Y cr Z cr The corresponding right camera C r Image pixels (u r ,v r The coordinates are:

[0169]

[0170] Step 3, Projector P j The coordinates in the pixel coordinate system are calculated as follows:

[0171] Given that the projector P is calculated from the first three sets of images j Transformation matrix between coordinate system and left phase coordinate system and projector P j The intrinsic and extrinsic parameter matrices. Projector P j Point P in the coordinate system p (X p ,Y p Z p It can be done through a projector P j The extrinsic parameter matrix from the left camera C l Coordinate system transformation:

[0172]

[0173] Among them, R p and T p It is a projector P j Relative to left camera C l The rotation and translation vectors.

[0174] Projector Pj Point P in the coordinate system p (X p ,Y p Z p Convert ) to a point (x) in the image coordinate system p ,y p The relationship is:

[0175]

[0176] Among them, f p For projector P j The focal length.

[0177] Projector P j Points in the image coordinate system (x p ,y p Convert to a point in pixel coordinates (u) p ,v p The relationship is:

[0178]

[0179] Among them, dx p and dy p Projector P j The physical dimensions of a pixel in the X and Y directions, (u 0p ,v 0p ) is the projector P j The position of the origin of the coordinate system in the pixel coordinate system.

[0180] Combining the above equations, we can obtain the three-dimensional point P. p (X p ,Y p Z p The corresponding projector P) j Image pixel coordinates (u 0p ,v 0p )for:

[0181]

[0182] Step 4: When the vertex of the calibration board is located in the binocular overlap region 3, the projector P... j Projection by pixel P p1 ,P p2 ,P p3 ,P p4 The target bounding box is formed by the left camera C in the calibration software. l The image shows coordinates P cl1 ,P cl2 ,P cl3 ,P cl4 The bounding box formed by the right camera C rThe image shows coordinates P cr1 ,P cr2 ,P cr3 ,P cr4 The target frame is formed to guide the placement of the calibration plate.

[0183] When the calibration board vertex is located in the non-overlapping region 2 of the left monocular camera, the right camera C r There is no corresponding point in the image, so it is determined by the left camera C. l and projector P j Similarly, when the calibration board vertex is located in the right monocular non-overlapping region 4, the left camera C is used for guidance. r and projector P j Provide guidance.

[0184] Once the calibration board is placed in the specified pose, a set of images for calibration is acquired. The calibration image sequence is the same as that in the pre-calibration process.

[0185] Please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the optimization of dual-target positioning accuracy using phase information, provided in an embodiment of this application.

[0186] Please refer to Figure 7 , Figure 7 This is a flowchart illustrating the optimization of dual-target positioning accuracy using phase information, provided in an embodiment of this application.

[0187] To ensure sufficiently high accuracy in bi-objective positioning, a phase consistency constraint is introduced into the bi-objective positioning optimization, as follows:

[0188] Step 1: Extract the calibration plate corner points using the full-brightness M0 image in each group of images, calculate the reprojection error using Zhang's calibration method, and construct the reprojection error constraint:

[0189]

[0190] Where, π l ,π r Left camera C l And right camera C r The projection function includes intrinsic parameters, extrinsic parameters, and a distortion model.

[0191] Step 2, using images M1-M1 from the 2nd to the 49th images in each group. 48 The relative phase is solved using the phase-shifting method, and the lateral and longitudinal absolute phases are obtained by expansion. For the left camera C... l A corner point P in the image l (u l ,v l Its transverse phase value is Φ lx (u l ,vl The longitudinal phase value is Φ. ly (u l ,v l The right camera C is found through corner point matching. r The corresponding corner point in the image is P. r (u r ,v r Its transverse phase value is Φ rx (u r ,v r The longitudinal phase value is Φ. ry (u r ,v r Using the phase difference between corresponding corner points, a phase consistency constraint error is constructed:

[0192]

[0193] Among them, E phase-x and E phase-y These represent the consistency constraint errors of the lateral phase and the longitudinal phase, respectively.

[0194] Combining reprojection error and phase consistency error, a dual-objective optimization objective function is constructed:

[0195] E stereo =λ1E phase-x +λ2E phase-y +λ3E reproj

[0196] Where λ1, λ2, and λ3 are weighting coefficients used to balance the influence of phase consistency constraints and reprojection constraints during the optimization process. The optimization variables include camera intrinsic parameters (focal length f, principal point (u0, v0), distortion coefficients k1, k2), extrinsic parameters (rotation matrix R, translation vector T), and corner coordinates. The optimization variable E is minimized using the Levenberg-Marquardt algorithm. stereo During the optimization process, the combined effect of lateral and longitudinal phase constraints ensures better phase consistency of corresponding corner points of the left and right cameras in both the lateral and longitudinal directions, thereby improving the S-axis performance of the binocular structured light system. b The calibration accuracy.

[0197] Furthermore, in order to integrate the left monocular structured light system S l Right monocular structured light system S r and binocular structured light system S b To address the calibration error, a joint optimization framework is proposed. This framework introduces single-target calibration error, dual-target calibration error, single / binocular parameter consistency constraints, and 3D point cloud consistency constraints, and performs joint optimization to reduce the parameter differences between single-target and dual-target calibration and the 3D point cloud consistency error.

[0198] Please refer to Figure 8 , Figure 8 The flowchart illustrates the monocular and binocular joint optimization framework provided in this application embodiment.

[0199] To clearly describe the joint optimization framework, the following symbols and definitions are added:

[0200] Monocular projection model:

[0201] Left camera C l +Projector P j Model: π lp (·)(Left camera C) l Parameter Θ1 + Projector P j Parameter Θ p ).

[0202] Right camera C r +Projector P j Model: π rp (·)(Right camera C) r Parameter Θ r +Projector P j Parameter Θ p ).

[0203] Three-dimensional point consistency region:

[0204] X, the corner point of the calibration plate observed jointly by binoculars and monoculars i The coordinates reconstructed by monocular and binocular cameras are consistent.

[0205] Parameter sharing constraints:

[0206] Camera parameters Θ in dual-target positioning l ,Θ r The parameters should be consistent with those in the single-target calibration.

[0207] like Figure 8 The errors in each part are described as follows:

[0208] Single target positioning error:

[0209] For monocular structured light systems, the monocular reprojection error (projection error between camera observation coordinates and monocular reconstructed points) and the monocular phase error (projector P) are combined. j (Constructing the single-view positioning error equation by ensuring consistency between the phase coordinates and the projection phase of the monocular reconstruction point.)

[0210] Left camera C l The calibration error of the monocular structured light system is:

[0211]

[0212] Right camera C rThe calibration error of the monocular structured light system is:

[0213]

[0214] in and These are 3D points reconstructed independently by a monocular system.

[0215] Bi-target positioning error:

[0216] Based on the aforementioned bi-target positioning error, we have:

[0217]

[0218] Monocular and binocular parameter consistency constraints:

[0219] Forced dual-target localization and single-target localization to share the same set of camera and projector parameters to avoid parameter inconsistencies. The parameter consistency error equation is as follows:

[0220]

[0221] 3D point cloud consistency constraints:

[0222] In the binocular and monocular co-observation area, i.e., binocular overlap region 3, the 3D point coordinates reconstructed by monocular and binocular cameras are forced to be consistent. The consistency error of the 3D point coordinates is:

[0223]

[0224] Joint optimization equation:

[0225] The above four factors are weighted and combined to form the overall objective function:

[0226] E total =λ1E mono-l +λ2E mono-r +λ3E param +λ4E point +λ5E stereo

[0227] Minimize E using the Levenberg-Marquardt optimization algorithm total This improves the calibration accuracy of monocular and binocular systems, and reduces the difference between monocular and binocular calibration parameters as well as the consistency error of 3D point clouds.

[0228] This embodiment projects the guide frame and the marked target frame in the camera image of the calibration software through a projector, which actively guides the pose of the calibration board during the calibration process. This simplifies the calibration conditions by eliminating the need for special devices and avoids the need for recalibration due to changes in the calibration space caused by changes in the position of the structured light system or the calibration device during the calibration process.

[0229] This embodiment ensures that the calibration area of ​​the calibration board covers the entire measurement space and the calibration board is in the optimal pose by dividing the calibration space and dynamically guiding the calibration board's pose, which significantly improves the consistency of calibration accuracy throughout the entire measurement space of the structured light system.

[0230] In this embodiment, a fully illuminated image and an coded image are projected onto the calibration board using a projector. Simultaneously, the calibration board image and the structured light coded image are acquired and used for feature point detection and decoding to obtain phase information, respectively. By using the phase consistency of left and right corresponding points as a constraint, a joint optimization equation for reprojection error and phase consistency error is constructed, which can significantly improve the calibration accuracy of the binocular camera.

[0231] This embodiment jointly optimizes single-target and dual-target positioning errors, which can significantly improve single-target positioning accuracy while reducing the difference between single-target and dual-target positioning errors. Introducing single- and dual-target parameter consistency constraints can significantly reduce the difference between single-target and dual-target positioning parameters, ensuring consistency between single-target and dual-target positioning parameters. Introducing 3D point cloud consistency constraints forces the 3D point coordinates of single-target and dual-target reconstructions to be consistent, significantly improving the point cloud consistency of the fusion region between single-target and dual-target reconstructions. Finally, constructing a single- and dual-target joint optimization equation can guarantee the consistency of single-target and dual-target positioning accuracy in a large field-of-view structured light system, as well as the consistency of single-target and dual-target reconstruction accuracy and the fusion degree of the reconstructed point cloud.

[0232] The following describes a projector-assisted binocular vision system calibration device provided in an embodiment of this application. The projector-assisted binocular vision system calibration device described below and the projector-assisted binocular vision system calibration method described above can be referred to in correspondence with each other.

[0233] Please refer to Figure 9 , Figure 9 This is a schematic diagram of a projector-assisted binocular vision system calibration device provided in an embodiment of this application.

[0234] In this embodiment, the device may include:

[0235] The pre-calibration module 100 is used to pre-calibrate the monocular and binocular structured light systems based on Zhang's calibration method to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the intrinsic and extrinsic parameters of the right monocular structured light system, and the intrinsic and extrinsic parameters of the binocular structured light system. The binocular structured light system includes a left camera, a right camera, and a projector.

[0236] The calibration board placement module 200 is used to calculate the calibration space based on the endo- and endo-parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. It then calculates the target bounding box based on the size of the calibration board and each calibration space. The target bounding box is used to guide the placement of the calibration board via a projector.

[0237] The image acquisition module 300 is used to control the projector to project a preset image onto the calibration board after the calibration board is placed, and to control the camera to acquire the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image;

[0238] The mono- and binocular joint optimization module 400 is used to perform joint optimization based on the reprojection error and phase consistency constraint error determined by the calibration board image to obtain the binocular calibration error.

[0239] The target optimization module 500 is used to perform joint optimization based on the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud to obtain the target calibration error;

[0240] The calibration processing module 600 is used for calibration processing based on the target calibration error.

[0241] Optionally, a calibration board placement module is used to calculate the calibration space based on the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; to divide each calibration space into regions based on the size of the calibration board, to obtain the corresponding calibration board placement pose; to calculate the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system based on the placement pose of each calibration board; and to connect the pixel coordinates of the four vertices of the calibration board to obtain the target bounding box.

[0242] This application also provides a binocular vision system calibration device; please refer to [reference needed]. Figure 10 , Figure 10 This is a schematic diagram of the structure of a binocular vision system calibration device provided in an embodiment of this application. The binocular vision system calibration device may include:

[0243] Memory, used to store computer programs;

[0244] The processor, when executing a computer program, can implement the steps of any of the projector-assisted binocular vision system calibration methods described above.

[0245] like Figure 10 The diagram shows the structural composition of a binocular vision system calibration device. The device may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other via the communication bus 13.

[0246] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0247] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the abnormal IP identification method.

[0248] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0249] Based on Zhang's calibration method, the monocular and binocular structured light systems were pre-calibrated to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. The binocular structured light system includes a left camera, a right camera, and a projector.

[0250] Based on the endo- and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, calibration spaces are calculated to obtain the calibration spaces of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. Target bounding boxes are then calculated based on the dimensions of the calibration board and each calibration space. These target bounding boxes are used to guide the placement of the calibration board via a projector.

[0251] After the calibration board is placed, the projector projects a preset image onto the calibration board, and the camera captures the image of the calibration board. The preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image.

[0252] The bi-target calibration error is obtained by jointly optimizing the reprojection error and phase consistency constraint error determined by the calibration board image.

[0253] The target calibration error is obtained by jointly optimizing the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of monocular and binocular parameters, and the consistency error of the 3D point cloud.

[0254] Calibration is performed based on the target calibration error.

[0255] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0256] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0257] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0258] Of course, it should be noted that, Figure 10 The structure shown does not constitute a limitation on the binocular vision system calibration device in the embodiments of this application. In practical applications, the binocular vision system calibration device may include... Figure 10 More or fewer components as shown, or combinations of certain components.

[0259] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of any of the above-described projector-assisted binocular vision system calibration methods.

[0260] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0261] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0262] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0263] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0264] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0265] The foregoing has provided a detailed description of a projector-assisted binocular vision system calibration method, binocular vision system calibration device, binocular vision system calibration equipment, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for calibrating a projector-assisted binocular vision system, characterized in that, include: Based on Zhang's calibration method, the monocular and binocular structured light systems were pre-calibrated to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector; Based on the endo- and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, calibration spaces are calculated to obtain the calibration spaces of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. A target frame is then calculated based on the dimensions of the calibration board and each calibration space. The target frame is used to guide the placement of the calibration board via a projector. After the calibration board is placed, the projector is controlled to project a preset image onto the calibration board, and the camera is controlled to capture the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image; The reprojection error and phase consistency constraint error determined by the calibration board image are jointly optimized to obtain the dual-target calibration error; The target calibration error is obtained by jointly optimizing the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud. Calibration processing is performed based on the target calibration error.

2. The binocular vision system calibration method according to claim 1, characterized in that, Based on Zhang's calibration method, the monocular and binocular structured light systems were pre-calibrated to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, including: Both the left and right cameras acquire pre-calibrated images; Based on the acquired pre-calibrated images, Zhang's calibration process is performed to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the intrinsic and extrinsic parameters of the right monocular structured light system, and the intrinsic and extrinsic parameters of the binocular structured light system.

3. The binocular vision system calibration method according to claim 2, characterized in that, Based on the endo- and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, calibration spaces are calculated to obtain the calibration spaces of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. Then, based on the size of the calibration board and each calibration space, a target bounding box is calculated, including: Based on the intrinsic and extrinsic parameters of the left monocular structured light system, the extrinsic and extrinsic parameters of the right monocular structured light system, and the extrinsic and extrinsic parameters of the binocular structured light system, calibration space calculations are performed to obtain the calibration space of the left monocular structured light system, the calibration space of the right monocular structured light system, and the calibration space of the binocular structured light system. Based on the size of the calibration plate, each calibration space is divided into regions to obtain the corresponding calibration plate placement pose; Based on the placement pose of each calibration board, the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system are calculated; The target bounding box is obtained by connecting the pixel coordinates of the four vertices of the calibration board.

4. The binocular vision system calibration method according to claim 3, characterized in that, The calibration plate is placed using a projector, including: The projector projects the target frame, and the camera image displays the target frame as an aid to guide the placement of the calibration plate.

5. The binocular vision system calibration method according to claim 4, characterized in that, Based on the reprojection error and phase consistency constraint error determined by the calibration board image, a joint optimization is performed to obtain the bi-target calibration error, including: The corner points of the calibration board are extracted based on the full-brightness image of the calibration board image, and the reprojection error is obtained by constructing the error of the corner points of the calibration board using Zhang's calibration method. The phase-shifting method is used to perform phase expansion on the images of the calibration board, excluding the fully bright images, to obtain the horizontal absolute phase and the vertical absolute phase. Errors are constructed based on the phase differences between corresponding corner points in the calibration plate, the lateral absolute phase, and the longitudinal absolute phase to obtain phase consistency constraint errors; The bi-target positioning error is obtained by minimizing the first preset weight, the reprojection error, and the phase consistency constraint error using the Levenberg-Marquardt algorithm.

6. The binocular vision system calibration method according to claim 5, characterized in that, The target calibration error is obtained by jointly optimizing the calibration errors of the left monocular system, the right monocular system, the binocular calibration error, the monocular and binocular parameter consistency error, and the 3D point cloud consistency error. This includes: Error construction processing is performed based on the corresponding monocular reprojection error and the corresponding monocular phase error to obtain the calibration error of the left monocular system and the calibration error of the right monocular system; Error construction processing is performed based on the intrinsic and extrinsic parameters of the monocular structured light system and the intrinsic and extrinsic parameters of the binocular structured light system to obtain the monocular and binocular parameter consistency error. The three-dimensional point coordinates of the binocular overlapping area between the monocular structured light system and the binocular structured light system are made consistent, and error construction processing is performed to obtain the three-dimensional point cloud consistency error. The target calibration error is obtained by minimizing the second preset weight, the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the binocular system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud using the Levenberg-Marquardt algorithm.

7. A calibration device for a projector-assisted binocular vision system, characterized in that, include: The pre-calibration module is used to pre-calibrate the monocular and binocular structured light systems based on Zhang's calibration method, and to obtain the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system; wherein, the binocular structured light system includes: a left camera, a right camera, and a projector. The calibration board placement module is used to calculate the calibration space based on the endo- and endo-parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system. It then calculates a target frame based on the size of the calibration board and each calibration space. The target frame is used to guide the placement of the calibration board via a projector. The image acquisition module is used to control the projector to project a preset image onto the calibration board after the calibration board is placed, and to control the camera to acquire the image of the calibration board; wherein, the preset image includes a full-brightness image, a horizontal coded stripe image, and a vertical coded stripe image; The monocular and binocular joint optimization module is used to perform joint optimization based on the reprojection error and phase consistency constraint error determined by the calibration board image to obtain the binocular calibration error; The target optimization module is used to perform joint optimization based on the calibration error of the left monocular system, the calibration error of the right monocular system, the calibration error of the dual-target system, the consistency error of the monocular and binocular parameters, and the consistency error of the 3D point cloud to obtain the target calibration error; The calibration processing module is used to perform calibration processing based on the target calibration error.

8. The binocular vision system calibration device according to claim 7, characterized in that, The calibration board placement module is specifically used to perform calibration space calculations based on the intrinsic and extrinsic parameters of the left monocular structured light system, the right monocular structured light system, and the binocular structured light system, respectively, to obtain the calibration space of the left monocular structured light system, the calibration space of the right monocular structured light system, and the calibration space of the binocular structured light system. Based on the size of the calibration plate, each calibration space is divided into regions to obtain the corresponding calibration plate placement pose; Based on the placement pose of each calibration board, the corresponding pixel coordinates of the four vertices of the calibration board in the camera pixel coordinate system and the projector pixel coordinate system are calculated; The target bounding box is obtained by connecting the pixel coordinates of the four vertices of the calibration board.

9. A binocular vision system calibration device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the binocular vision system calibration method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the binocular vision system calibration method as described in any one of claims 1 to 7.

Images