A method, device and computer equipment for correcting a binocular device

Abstract

The application relates to a calibration method, device and computer equipment of a binocular device. The method comprises the following steps: acquiring and determining a rotation matrix, an offset matrix and an internal parameter of each camera based on a calibration board image pair; obtaining a reference image of the calibration board on the premise that the lens of a reference camera is parallel to the calibration board; determining the external parameter of the reference camera based on the reference image and the internal parameter of the reference camera; determining the transformation matrix of the reference camera and the transformation matrix of a non-reference camera based on the rotation matrix and the offset matrix between each camera of the binocular device and the external parameter of the reference camera; and calibrating the binocular device to obtain a binocular device with a fixed visual axis. The method can solve the problem that the existing calibration method of the binocular device cannot make the data of edge uniformity and unit relative distortion meet the test standard when the binocular endoscope is tested for edge uniformity or unit relative distortion, and the test card has trapezoidal distortion in the image.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a method, apparatus and computer device for calibrating a binocular device. Background Technology

[0002] With the development of technology, computer vision technology has advanced rapidly. As a result, binocular vision devices are widely used. In the application of binocular vision devices, calibration is necessary to ensure optimal display quality.

[0003] Existing binocular camera calibration methods primarily rely on stereo calibration results to obtain the extrinsic parameters of each camera, and then use these parameters to derive the binocular camera's rotation matrix. Calibration typically involves keeping one camera stationary while rotating the other by the angle of the rotation matrix, or rotating each camera along half the angle of the rotation matrix in opposite directions along the rotation vector to make the optical axes of the two cameras parallel. However, this method results in inconsistencies between the optical and visual axes of the image, potentially impacting product quality. For example, with binocular endoscopes, single-channel images require tests such as relative distortion and edge uniformity. If the visual and optical axes are not parallel, after adjusting the distance and angle between the test chart and the camera, the displayed test chart image may exhibit trapezoidal distortion, easily leading to data that fails to meet testing standards for edge uniformity and relative distortion.

[0004] Regarding the existing calibration methods for binocular devices, when calibrating binocular endoscopes, there is an inconsistency between the optical axis and visual axis of the image tested by the endoscope. When performing edge uniformity tests or unit relative distortion tests, after adjusting the position of the test card and the equipment according to the test standards, the test card exhibits trapezoidal distortion in the image, causing the data for edge uniformity, unit relative distortion, and other tests to fail to meet the test standards. Currently, no effective solution has been proposed. Summary of the Invention

[0005] Therefore, it is necessary to provide a calibration method, apparatus, and computer device for binocular devices to address the aforementioned technical problems.

[0006] Firstly, this application provides a calibration method for a binocular device. The method includes the following steps:

[0007] Acquire multiple calibration board image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device based on the multiple calibration board image pairs;

[0008] A reference camera in the binocular device is determined, and with the lens of the reference camera parallel to the calibration board, multiple reference images of the calibration board are obtained using the reference camera. Based on the multiple reference images and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined.

[0009] Based on the rotation and offset matrices between the cameras of the binocular device, and the extrinsic parameters of the reference camera, the transformation matrix of the reference camera and the transformation matrix of the non-reference camera are determined.

[0010] Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, the individual cameras of the binocular device are calibrated to obtain a binocular device with a fixed line of sight.

[0011] In one embodiment, acquiring multiple calibration board image pairs captured by the binocular device at different shooting angles, and determining the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device based on the multiple calibration board image pairs, includes the following steps:

[0012] The calibration board is photographed by each camera of the binocular device at different shooting angles and shooting distances to obtain multiple pairs of images of the calibration board;

[0013] Based on multiple calibration board image pairs, a preset binocular calibration algorithm is used to determine the rotation matrix, the offset matrix, and the intrinsic parameters of each camera in the binocular device.

[0014] In one embodiment, determining a reference camera in the binocular device and obtaining multiple reference images of the calibration board using the reference camera, provided that the lens of the reference camera is parallel to the calibration board, includes the following steps:

[0015] Select any camera in the binocular device as the reference camera of the binocular device;

[0016] Adjust the reference camera until its lens is parallel to the calibration plate to obtain the adjusted reference camera;

[0017] The calibration board is photographed using the adjusted reference camera to obtain multiple reference images of the calibration board.

[0018] In one embodiment, determining the extrinsic parameters of the reference camera based on multiple reference images and the intrinsic parameters of the reference camera includes the following steps:

[0019] Feature extraction is performed on each of the reference images to obtain feature points in each of the reference images;

[0020] Based on the feature points in each of the reference images, feature point matching is performed between the reference images and the calibration board to obtain the feature point matching results between multiple reference images and the calibration board;

[0021] Based on the matching results of feature points between multiple reference images and the calibration board, as well as the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined.

[0022] In one embodiment, determining the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and offset matrix between the cameras of the stereo device, and the extrinsic parameters of the reference camera, includes the following steps:

[0023] Based on the rotation matrix between the cameras of the binocular device and the extrinsic parameters of the reference camera, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined.

[0024] Based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the stereo device, the transformation matrix of the reference camera and the transformation matrix of the non-reference camera are determined.

[0025] In one embodiment, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera includes the following steps:

[0026] Based on the rotation matrix between the cameras of the binocular device and the extrinsic parameters of the reference camera, the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of each camera of the binocular device are determined.

[0027] Based on the rotation angle of the reference camera relative to the fixed line of sight, and the rotation angles of each camera in the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined.

[0028] In one embodiment, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to a fixed line of sight and the rotation angles of the individual cameras of the binocular device includes the following steps:

[0029] The rotation matrix of the reference camera is determined based on the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of the reference camera in the binocular device.

[0030] The rotation matrix of the non-reference camera is determined based on the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of each camera in the binocular device.

[0031] In one embodiment, determining the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to a fixed line of sight and the rotation angles of the respective cameras in the binocular device includes the following steps:

[0032] The rotation angle of the non-reference camera relative to the fixed line of view is determined based on the rotation angle of the reference camera relative to the fixed line of view and the rotation angle of each camera in the binocular device.

[0033] The rotation matrix of the non-reference camera is determined based on the rotation angle of the non-reference camera relative to the fixed line of sight.

[0034] Secondly, this application also provides a calibration device for a binocular device. The device includes:

[0035] The intrinsic and extrinsic parameter determination module is used to acquire multiple calibration board image pairs captured by the binocular device at different shooting angles, and based on the multiple calibration board image pairs, determine the rotation matrix, offset matrix and intrinsic parameters of each camera of the binocular device;

[0036] The reference camera extrinsic parameter determination module is used to determine the reference camera in the binocular device, and, under the premise that the lens of the reference camera is parallel to the calibration board, to obtain multiple reference images of the calibration board using the reference camera, and to determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera.

[0037] The transformation matrix determination module is used to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and offset matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera.

[0038] And a correction module, used to correct each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, to obtain a binocular device with a fixed line of sight.

[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the calibration method for the binocular device described in the first aspect.

[0040] The aforementioned binocular device calibration method, apparatus, and computer equipment determine the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device by utilizing multiple calibration plate image pairs captured by the binocular device at different angles. Then, a reference camera is used to capture multiple reference images of the calibration plate with a fixed viewing axis. Using these reference images and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera in the binocular device are determined. Finally, based on the rotation matrix and offset matrix between the two cameras in the binocular device, and the absolute extrinsic parameters of the reference camera, the various variable parameters of each camera in the binocular device relative to the fixed viewing axis are determined. The matrix is ​​changed, and then, based on the transformation matrix of each camera in the binocular device, each camera in the binocular device is calibrated to obtain a binocular device with a fixed visual axis. This ensures that the optical axis and visual axis of the image of the binocular device are consistent. This solves the problem that when calibrating binocular endoscopes based on existing binocular device calibration methods, the optical axis and visual axis of the images tested by the endoscope are inconsistent. When performing edge uniformity tests or unit relative distortion tests, after adjusting the position of the test card and the device according to the test standard, the test card exhibits trapezoidal distortion in the image, causing the data for edge uniformity, unit relative distortion, and other tests to fail to meet the test standards.

[0041] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0042] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1 A hardware structure block diagram of a terminal for a binocular device calibration method provided in an embodiment of this application;

[0044] Figure 2 A flowchart illustrating a calibration method for a binocular device provided in an embodiment of this application;

[0045] Figure 3 A flowchart illustrating a preferred embodiment of the calibration method for a binocular device provided in this application;

[0046] Figure 4 This is a structural block diagram of a calibration device for a binocular device provided in an embodiment of this application. Detailed Implementation

[0047] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0048] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0049] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal of the binocular device calibration method in this embodiment. (See diagram for example.) Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the binocular device calibration method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0052] This embodiment provides a calibration method for a binocular device. Figure 2 This is a flowchart of the calibration method for the binocular device in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0053] Step S210: Obtain multiple calibration board image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device based on the multiple calibration board image pairs.

[0054] The aforementioned binocular device can be a binocular imaging device, such as a binocular endoscope or binocular microscope. The binocular device includes two cameras, left and right, each camera including a lens, and the relative position between the two cameras is fixed. The aforementioned calibration plate can be a flat plate with a fixed-spacing pattern array, wherein the fixed-spacing pattern can be a dot array, a checkerboard pattern, or other patterns with obvious corner features. It should be noted that, to ensure that the captured image of the calibration plate contains a sufficient number of feature points, the corner features in the calibration plate need to reach a preset number of corner points. Specifically, the preset number of corner points can be set according to specific circumstances. For example, the preset number of corner points can be 54.

[0055] The aforementioned acquisition of multiple calibration board image pairs captured by the binocular device at different shooting angles can be achieved by simultaneously capturing images of the calibration board from different shooting angles using various cameras of the binocular device. It should be noted that when capturing images of the calibration board using the binocular device, it is necessary to ensure that the images captured by the binocular device are within the depth of field, and the image sharpness of the acquired calibration board image pairs must meet preset sharpness requirements. By acquiring the calibration board image pairs, the extrinsic parameters (i.e., rotation matrices and translation vectors relative to a certain world coordinate system) and intrinsic parameters of each camera in the binocular device can be calculated. Using the extrinsic and intrinsic parameters of each camera in the binocular device, the rotation matrix and offset matrix between the various cameras can be determined.

[0056] Furthermore, the aforementioned rotation matrix can represent the rotation relationship between the two camera coordinate systems of a stereo device. Specifically, it can be the rotation matrices of each camera rotating to the target coordinate system. For example, if a stereo device has a left camera and a right camera, the rotation matrix between the left and right cameras includes a first rotation matrix from the left camera to the target coordinate system and a second rotation matrix from the right camera to the target coordinate system. That is, the rotation from the left camera coordinate system to the target coordinate system is the first rotation matrix, and the rotation from the right camera coordinate system to the target coordinate system is the second rotation matrix. The aforementioned offset matrix is ​​usually represented as a translation vector, indicating the translation relationship between the two camera coordinate systems.

[0057] The intrinsic parameters of the aforementioned camera may include the camera's focal length, principal point position, distortion coefficient, etc.

[0058] This step determines the positional relationship between the two cameras of the stereo device by determining the rotation matrix and offset matrix between each camera.

[0059] Step S220: Determine the reference camera in the binocular device, and under the premise that the lens of the reference camera is parallel to the calibration board, use the reference camera to obtain multiple reference images of the calibration board, and determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera.

[0060] In this step, the reference camera in the binocular device can be any one camera from the device, with the other serving as a non-reference camera. It should be noted that when the reference camera's lens is parallel to the calibration plate, its shooting direction is perpendicular to the plane of the calibration plate. In this case, the reference image of the calibration plate captured by the reference camera will not exhibit perspective distortion. Therefore, the calibration plate can be used to pre-set a fixed viewing axis. When the calibration plate is fixed in position, adjusting the reference camera's lens to be parallel to the calibration plate ensures that the reference camera's viewing axis is a fixed viewing axis (i.e., perpendicular to the calibration plate from the center of the camera lens). During absolute extrinsic parameter calibration, if there is rotation in a non-planar direction, the optical axis and viewing axis will not be parallel. It should be noted that when obtaining multiple reference images of the calibration plate using the reference camera, it is only necessary to keep the reference camera's lens parallel to the calibration plate. The distance between the reference camera and the calibration plate can vary, as long as the calibration plate remains within the depth of field of the reference camera, and the captured reference images meet the preset sharpness requirements.

[0061] The above-mentioned determination of the extrinsic parameters of the reference camera based on multiple reference images can be achieved by extracting features from each reference image to obtain feature points in each image. Then, based on the feature points in each reference image and the intrinsic parameters of the reference camera, the Zhang Zhengyou calibration algorithm is applied to determine the extrinsic parameters of the reference camera. The aforementioned extrinsic parameters of the reference camera are its absolute extrinsic parameters, specifically the rotation matrix and translation vector of the reference camera relative to the world coordinates of the calibration card.

[0062] Step S230: Based on the rotation matrix and offset matrix between the cameras of the stereo device, and the extrinsic parameters of the reference camera, determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0063] The transformation matrix of the aforementioned reference camera represents the transformation relationship required when transforming the reference camera to have a fixed line of sight. Similarly, the transformation matrix of the aforementioned non-reference camera represents the transformation relationship required when transforming a non-reference camera to have a fixed line of sight.

[0064] In this step, the transformation matrix of the reference camera and the transformation matrix of the non-reference camera are determined based on the rotation matrix and offset matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera. This can be achieved by determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix and offset matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera, and then determining the transformation matrix of each camera of the stereo device based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the stereo device.

[0065] Step S240: Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, the individual cameras of the stereo device are calibrated to obtain a stereo device with a fixed line of sight.

[0066] Steps S210 to S240 above involve using multiple calibration board image pairs captured by the binocular device at different angles to determine the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device. Then, a reference camera is used to capture multiple reference images of the calibration board with a fixed viewing axis. Using these reference images and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera in the binocular device are determined. Finally, based on the rotation matrix and offset matrix between the two cameras in the binocular device, and the absolute extrinsic parameters of the reference camera, the transformation matrices of the two cameras in the binocular device corresponding to the fixed viewing axis are determined. Then, based on the transformation matrix of each camera in the binocular device, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis, so that the optical axis and visual axis of the image of the binocular device are consistent. This solves the problem that when calibrating binocular endoscopes based on existing binocular device calibration methods, the optical axis and visual axis of the images tested by the endoscope are inconsistent. When performing edge uniformity tests or unit relative distortion tests, after adjusting the position of the test card and the device according to the test standard, the test card has trapezoidal distortion in the image, which causes the data of edge uniformity, unit relative distortion and other tests to not meet the test standards.

[0067] In one embodiment, based on step S210, multiple calibration board image pairs captured by the binocular device at different shooting angles are acquired, and based on the multiple calibration board image pairs, the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device are determined, including:

[0068] Step S212: Use the cameras of the binocular device to take pictures of the calibration board at different shooting angles and shooting distances to obtain multiple calibration board image pairs.

[0069] The above-mentioned method of using the various cameras of a binocular device to capture images of the calibration board at different shooting angles and distances to obtain multiple calibration board image pairs can be achieved by adjusting the shooting angle and position of the binocular device so that the various cameras of the device simultaneously capture images of the calibration board at different shooting angles and distances, thus obtaining multiple calibration board image pairs. These calibration board image pairs are the matching results of calibration board images captured simultaneously by different cameras.

[0070] Step S214: Based on multiple calibration board image pairs, use a preset dual-camera calibration algorithm to determine the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device.

[0071] The aforementioned pre-defined bi-target calibration algorithm can be one or more of the following algorithms: Zhang's calibration method, ICP (Iterative Closest Point) algorithm, DLT (Direct Linear Transform) algorithm, etc.

[0072] Specifically, the above-mentioned determination of the rotation matrix, offset matrix, and intrinsic parameters of each camera in the stereo device based on multiple calibration board image pairs and using a preset stereo calibration algorithm can be achieved by extracting feature points from each image in the multiple calibration board image pairs, then performing feature point matching on each calibration board image pair based on the extracted feature points to obtain the feature point matching results of each calibration board image pair, and finally using the preset stereo calibration algorithm to determine the rotation matrix, offset matrix, and intrinsic parameters of each camera in the stereo device based on the feature point matching results of each calibration board image pair.

[0073] Steps S212 to S214 above involve using the cameras of the binocular device to capture images of the calibration board at different shooting angles and distances, resulting in multiple calibration board image pairs. Then, based on the calibration board image pairs, the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device are determined. Determining the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device facilitates the subsequent determination of the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0074] Specifically, in one embodiment, step S220 above, which involves determining a reference camera in the binocular device and obtaining multiple reference images of the calibration board using the reference camera, with the lens of the reference camera parallel to the calibration board, includes:

[0075] Step S221: Select any camera in the stereo device as the reference camera of the stereo device.

[0076] Step S222: Adjust the reference camera until its lens is parallel to the calibration plate to obtain the adjusted reference camera.

[0077] Step S223: Use the adjusted reference camera to take pictures of the calibration board to obtain multiple reference images of the calibration board.

[0078] It should be noted that when using the reference camera to photograph the calibration board, it is only necessary to ensure that the lens of the reference camera is parallel to the calibration board, and the distance between the reference camera and the calibration board can be changed.

[0079] Steps S221 to S223 above involve selecting a reference camera in the binocular device and adjusting the reference camera so that its lens is parallel to the calibration board. This facilitates obtaining a reference image of the calibration board while the lens of the reference camera is parallel to the calibration board, and makes it easier to obtain the extrinsic parameters of the reference camera using the reference image of the calibration board.

[0080] In another embodiment, in step S220 above, determining the extrinsic parameters of the reference camera based on multiple reference images and the intrinsic parameters of the reference camera includes:

[0081] Step S224: Extract features from each reference image to obtain feature points in each reference image.

[0082] The aforementioned method for determining the feature points of the reference image can be a feature point detection algorithm such as SIFT (Scale Invariant Feature Transform) or Harris algorithm, or other feature point detection algorithms, as long as the feature points of each reference image can be determined. This embodiment does not specifically limit the method of obtaining the feature points of the reference image.

[0083] Step S225: Based on the feature points in each reference image, perform feature point matching between the reference image and the calibration board to obtain the feature point matching results between each reference image and the calibration board.

[0084] Step S226: Based on the matching results of feature points between each reference image and the calibration board, and the intrinsic parameters of the reference camera, determine the extrinsic parameters of the reference camera.

[0085] Because the shape of the calibration board set is fixed, and each feature point in the calibration board has known world coordinates, a correspondence can be established between each feature point in the calibration board and the feature points in the reference image based on the matching results between the feature points of each reference image and the calibration board, as well as the position of each feature point in the calibration board in the reference image. Then, the extrinsic parameters between the reference camera and the calibration board can be solved using the PNP (Perspective-n-Point) algorithm, with the calibration board coordinate system as the world coordinate system, to obtain the extrinsic parameters of the reference camera.

[0086] Based on Zhang Zhengyou's monocular camera calibration method, after calculating the homography matrix, the extrinsic parameters (rotation + translation matrices) of each image are calculated using the intrinsic parameter data obtained in the binocular calibration. The extrinsic parameters of the images mentioned above can be the absolute extrinsic parameters of the images. Because the requirement for image capture is that the lens of the reference camera is parallel to the calibration board, the rotation matrix parameters in the absolute extrinsic parameters obtained from different reference images are relatively consistent. After obtaining each absolute extrinsic parameter, an optimization method is used to obtain the rotation matrix in the optimal absolute extrinsic parameters.

[0087] Steps S224 to S226 above involve extracting features from each reference image to obtain feature points in each reference image. Then, using the feature points of each reference image and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined. By determining the extrinsic parameters of the reference camera, it is easier to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the extrinsic parameters of the reference camera.

[0088] In one embodiment, based on step S230, the transformation matrix of the reference camera and the transformation matrix of the non-reference camera are determined based on the rotation matrix and offset matrix between the cameras of the stereo device, and the extrinsic parameters of the reference camera, including:

[0089] Step S232: Based on the rotation matrices between the cameras of the stereo device and the extrinsic parameters of the reference camera, determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0090] The aforementioned determination of the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera can be based on the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera to determine the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera of the stereo device, and then determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0091] The rotation matrix of the reference camera can be a rotation matrix that rotates the reference camera to a fixed viewing axis. Similarly, the rotation matrix of the non-reference camera can be a rotation matrix that rotates the non-reference camera to a fixed viewing axis.

[0092] Step S234: Based on the rotation matrix of the reference camera, the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the stereo device, determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0093] Given the offset matrix, based on the line alignment correction principle in epipolar correction, the line alignment matrix is ​​obtained through the offset matrix. This matrix is ​​then applied to the rotation matrices of the reference camera and the non-reference camera to obtain the transformation matrices of the reference camera and the non-reference camera.

[0094] Steps S232 to S234 above determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera by using the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera. Then, based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, as well as the offset matrix between the cameras of the stereo device, the transformation matrix of each camera of the stereo device is determined. This facilitates the subsequent correction of the stereo device based on the transformation matrix of each camera, so that the stereo device has a fixed line of sight.

[0095] In another embodiment, step S232 above, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrices between the cameras of the binocular device and the extrinsic parameters of the reference camera, includes:

[0096] Step S2322: Based on the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera, determine the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angle of each camera of the stereo device.

[0097] The rotation angle of the reference camera relative to the fixed viewing axis can be the rotation angle of the reference camera to the fixed viewing axis in various directions. These directions can be the x-axis, y-axis, and z-axis directions of the world coordinate system. The rotation angle of each camera in the binocular device can be the rotation angle of each camera coordinate system of the binocular device to the target coordinate system. For example, if the binocular device has a left camera and a right camera, the rotation angles of the left and right cameras include a first rotation angle from the left camera to the target coordinate system and a second rotation angle from the right camera to the target coordinate system. The first rotation angle is the rotation angle from the left camera coordinate system to the target coordinate system in each direction. The second rotation angle is the rotation angle from the right camera coordinate system to the target coordinate system in each direction. It should be noted that the target coordinate system can be the coordinate system corresponding to the minimum angle when the left and right paths are aligned with the relative rotation coordinate systems, or a fixed coordinate system. For example, a coordinate system where the optical axis and the viewing axis are parallel.

[0098] The calculation process for determining the rotation angle of the reference camera relative to the fixed line of sight, based on the extrinsic parameters of the reference camera, is as follows:

[0099]

[0100] Where R is the rotation matrix in the extrinsic parameters of the reference camera, r ab Let r represent the elements of the rotation matrix in the extrinsic parameters of the reference camera, where 'a' represents the 'a'-th row of the rotation matrix in the extrinsic parameters of the reference camera, and 'b' represents the 'b'-th column of the rotation matrix in the extrinsic parameters of the reference camera. For example, r 11 This represents the first row and first column of the rotation matrix in the extrinsic parameters of the reference camera.

[0101] The Euler angle biases of the rotation matrix in the x, y, and z axes of the reference camera's extrinsic parameters can be expressed as:

[0102]

[0103] in, Let θ be the rotation angle of the reference camera about the x-axis relative to the fixed line of view, and let θ be the rotation angle of the reference camera about the y-axis relative to the fixed line of view. The reference camera's rotation angle around the z-axis relative to the fixed line of sight.

[0104] The rotation matrix in the extrinsic parameters of the reference camera, expressed in terms of rotation angles in three directions, is as follows:

[0105]

[0106] Solving the Euler angle rotation matrix formula yields the rotation angles of the reference camera relative to a fixed line of sight in different directions. The results are as follows:

[0107]

[0108] Given a fixed rotation matrix between the cameras of a stereo camera, a first rotation matrix and a second rotation matrix can be determined based on the rotation matrix between the cameras of the stereo camera. Given a fixed rotation matrix and a second rotation matrix, the rotation angles of the reference camera relative to the fixed line of sight in different directions can be determined by using the first rotation matrix. Based on the first rotation matrix, the first rotation angles of the left camera in the stereo camera relative to the target coordinate system in different directions can be determined. Based on the second rotation matrix, the rotation angles of the right camera in the stereo camera relative to the target coordinate system in different directions can be determined.

[0109] Step S2324: Based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device, determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0110] Steps S2322 to S2324 above determine the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device. Then, based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined. This facilitates the subsequent correction of each camera in the binocular device based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, thus obtaining a binocular device with a fixed viewing axis.

[0111] Further, in one embodiment, step S2324 above, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference cameras based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device, includes:

[0112] Step S1: Determine the rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angle of the reference camera in the binocular device.

[0113] Given a fixed rotation angle of the reference camera relative to a fixed viewing axis in different directions, the reference camera can be rotated around the x-axis and y-axis according to the fixed viewing axis, while rotating around the z-axis in the original direction (assuming the reference camera lens is parallel to the calibration plate, rotation around the z-axis only affects the rotation in the planar direction; to reduce clipping, the rotation in the planar direction is not considered when correcting the rotation matrix), thus obtaining the rotation matrix of the reference camera. The specific calculation process is as follows:

[0114]

[0115] Where R′ is the rotation matrix of the reference camera. This is the rotation angle around the z-axis in the first rotation angle.

[0116] In this embodiment, the left camera is used as the reference image and the right camera is used as the non-reference image for calculation. Alternatively, the right image can be used as the reference image and the left image as the non-reference image. This embodiment does not make a specific limitation here, and its calculation method is the same as the method of using the left image as the reference image.

[0117] Step S2: Based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device, determine the rotation matrix of the non-reference camera.

[0118] The process of calculating the rotation matrix of the non-reference camera is as follows:

[0119]

[0120] Where, r r ′ is the rotation matrix of the non-reference camera. θ is the rotation angle of the non-reference camera about the x-axis relative to the fixed line of sight. x The rotation angle of the non-reference camera about the y-axis relative to the fixed line of sight. This is the rotation angle around the z-axis in the second rotation angle.

[0121] In this embodiment, the left camera is used as the reference image and the right camera is used as the non-reference image for calculation. Alternatively, the right image can be used as the reference image and the left image as the non-reference image. This embodiment does not make a specific limitation here, and its calculation method is the same as the method of using the left image as the reference image.

[0122] Steps S1 to S2 above determine the rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angle of the reference camera in the binocular device. They also determine the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angles of each camera in the binocular device. The determination of the rotation matrices of the reference camera and the non-reference camera facilitates the subsequent calibration of each camera in the binocular device based on the rotation matrices of the reference camera and the non-reference camera, thus obtaining a binocular device with a fixed viewing axis.

[0123] Specifically, in one embodiment, based on step S2, based on the rotation angle of the reference camera relative to the fixed line of sight and the rotation angles of each camera in the binocular device, the rotation matrix of the non-reference camera is determined, including:

[0124] Step S22: Based on the rotation angle of the reference camera relative to the fixed axis of view and the rotation angles of each camera in the binocular device, determine the rotation angle of the non-reference camera relative to the fixed axis of view.

[0125] The calculation process for the rotation angle of the non-reference camera relative to the fixed line of sight is as follows:

[0126]

[0127] in, Let be the rotation angle about the x-axis in the second rotation angle. This is the rotation angle around the x-axis in the first rotation angle.

[0128] θ x =θ r +(θ-θ l )

[0129] Where, θ r Let θ be the rotation angle about the y-axis in the second rotation angle. l This is the rotation angle around the y-axis in the first rotation angle.

[0130]

[0131] in, The rotation angle of the non-reference camera around the z-axis relative to the fixed line of sight.

[0132] Step S24: Determine the rotation matrix of the non-reference camera based on the rotation angle of the non-reference camera relative to the fixed viewing axis.

[0133] Steps S22 to S24 above determine the rotation angle of the non-reference camera relative to the fixed visual axis based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device. Then, based on the rotation angle of the non-reference camera relative to the fixed visual axis and the rotation angle of the non-reference camera in the binocular device, the rotation matrix of the non-reference camera is determined. Confirmation of the rotation matrix of the non-reference camera facilitates subsequent calibration of each camera in the binocular device to obtain a binocular device with a fixed visual axis.

[0134] The present embodiment will now be described and illustrated through preferred embodiments.

[0135] Figure 3 This is a flowchart of a calibration method for a binocular device provided in a preferred embodiment of this application. Figure 3 As shown, the calibration method for this binocular device includes the following steps:

[0136] Step S301: Use the cameras of the binocular device to take pictures of the calibration board at different shooting angles and shooting distances to obtain multiple calibration board image pairs;

[0137] Step S302: Based on multiple calibration board image pairs, use a preset binocular calibration algorithm to determine the rotation matrix, offset matrix and intrinsic parameters of each camera in the binocular device.

[0138] Step S303: Determine the reference camera in the binocular device, and under the premise that the lens of the reference camera is parallel to the calibration board, use the reference camera to obtain multiple reference images of the calibration board, and determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera.

[0139] Step S304: Based on the rotation matrix between the cameras of the binocular device and the extrinsic parameters of the reference camera, determine the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angle of each camera of the binocular device.

[0140] Step S305: Determine the rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed viewing axis;

[0141] Step S306: Based on the rotation angle of the reference camera relative to the fixed viewing axis and the rotation angle of each camera in the binocular device, determine the rotation matrix of the non-reference camera.

[0142] Step S307: Based on the rotation matrix of the reference camera, the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the stereo device, determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0143] Step S308: Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, the individual cameras of the stereo device are calibrated to obtain a stereo device with a fixed line of sight.

[0144] Steps S301 to S308 above involve using multiple calibration board image pairs captured by the binocular device at different angles to determine the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device. Then, a reference camera is used to capture images of the calibration board with a fixed viewing axis, obtaining multiple reference images. The extrinsic parameters of the reference camera in the binocular device are determined using the reference images and the intrinsic parameters of the reference camera. Finally, based on the rotation matrix and offset matrix between the two cameras in the binocular device, and the extrinsic parameters of the reference camera, the transformation moments of the two cameras in the binocular device relative to the fixed viewing axis are determined. Then, based on the transformation matrix of each camera in the binocular device, each camera is calibrated to obtain a binocular device with a fixed visual axis, making the optical axis and visual axis of the image of the binocular device consistent. This solves the problem that when calibrating binocular endoscopes based on existing binocular device calibration methods, the optical axis and visual axis of the images tested by the endoscope are inconsistent. When performing edge uniformity tests or unit relative distortion tests, after adjusting the position of the test card and the device according to the test standard, the test card has trapezoidal distortion in the image, which causes the data of edge uniformity, unit relative distortion and other tests to not meet the test standard.

[0145] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0146] Based on the same inventive concept, this embodiment also provides a correction device for a binocular device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0147] In one embodiment, Figure 4 This is a structural block diagram of a binocular device calibration apparatus provided in an embodiment of this application, as shown below. Figure 4 As shown, the calibration device for the binocular device includes:

[0148] The intrinsic and extrinsic parameter determination module 42 is used to acquire multiple calibration board image pairs captured by the binocular device at different shooting angles, and based on the multiple calibration board image pairs, determine the rotation matrix, offset matrix and intrinsic parameters of each camera of the binocular device;

[0149] The reference camera extrinsic parameter determination module 44 is used to determine the reference camera in the binocular device, and, under the premise that the lens of the reference camera is parallel to the calibration board, to obtain multiple reference images of the calibration board using the reference camera, and to determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera.

[0150] The transformation matrix determination module 46 is used to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and offset matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera.

[0151] And a correction module 48, used to correct each camera of the stereo device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, to obtain a stereo device with a fixed line of sight.

[0152] The aforementioned binocular device calibration device determines the rotation matrix, offset matrix, and intrinsic parameters of each camera in the binocular device by utilizing multiple calibration plate image pairs captured by the binocular device at different angles. Then, it captures multiple reference images of the calibration plate with a fixed viewing axis using a reference camera. Using these reference images and the intrinsic parameters of the reference camera, it determines the extrinsic parameters of the reference camera in the binocular device. Finally, based on the rotation matrix and offset matrix between the two cameras of the binocular device, and the absolute extrinsic parameters of the reference camera, it determines the transformation matrices of the two cameras in the binocular device corresponding to the fixed viewing axis. Subsequently, based on the transformation matrix of each camera in the binocular device, each camera is calibrated to obtain a binocular device with a fixed visual axis. This ensures that the optical axis and visual axis of the binocular device's image are aligned, solving the problem that when calibrating a binocular endoscope using existing binocular device calibration methods, the optical axis and visual axis of the endoscope test image are inconsistent. Furthermore, when performing edge uniformity tests or unit relative distortion tests, after adjusting the position of the test card and device according to the test standards, the test card exhibits trapezoidal distortion in the image, causing the data for edge uniformity, unit relative distortion, and other tests to fail to meet the test standards.

[0153] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0154] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the binocular device calibration methods described in the above embodiments.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A calibration method for a binocular device, characterized in that, The method includes: Acquire multiple calibration board image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device based on the multiple calibration board image pairs; A reference camera in the binocular device is determined, and with the lens of the reference camera parallel to the calibration board, multiple reference images of the calibration board are obtained using the reference camera. Based on the multiple reference images and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined. Based on the rotation matrices between the cameras of the stereo device and the extrinsic parameters of the reference camera, the rotation matrices of the reference camera and the non-reference camera are determined; based on the rotation matrices of the reference camera and the non-reference camera, and the offset matrices between the cameras of the stereo device, the transformation matrices of the reference camera and the non-reference camera are determined; the rotation matrix of the reference camera is the rotation matrix for rotating the reference camera to a fixed viewing axis; the rotation matrix of the non-reference camera is the rotation matrix for rotating the non-reference camera to a fixed viewing axis. Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, the individual cameras of the binocular device are calibrated to obtain a binocular device with a fixed line of sight.

2. The calibration method for a binocular device according to claim 1, characterized in that, The step of acquiring multiple calibration board image pairs captured by the binocular device at different shooting angles, and determining the rotation matrix, offset matrix, and intrinsic parameters of each camera of the binocular device based on the multiple calibration board image pairs, includes: The calibration board is photographed by each camera of the binocular device at different shooting angles and shooting distances to obtain multiple pairs of images of the calibration board; Based on multiple calibration board image pairs, a preset binocular calibration algorithm is used to determine the rotation matrix, the offset matrix, and the intrinsic parameters of each camera in the binocular device.

3. The calibration method for a binocular device according to claim 1, characterized in that, The step of determining a reference camera in the binocular device, and obtaining multiple reference images of the calibration board using the reference camera, provided that the lens of the reference camera is parallel to the calibration board, includes: Select any camera in the binocular device as the reference camera of the binocular device; Adjust the reference camera until its lens is parallel to the calibration plate to obtain the adjusted reference camera; The calibration board is photographed using the adjusted reference camera to obtain multiple reference images of the calibration board.

4. The calibration method for a binocular device according to claim 1, characterized in that, Determining the extrinsic parameters of the reference camera based on multiple reference images and the intrinsic parameters of the reference camera includes: Feature extraction is performed on each of the reference images to obtain feature points in each of the reference images; Based on the feature points in each of the reference images, feature point matching is performed between the reference images and the calibration board to obtain the feature point matching results between each of the reference images and the calibration board. Based on the matching results of feature points between each of the reference images and the calibration board, and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined.

5. The calibration method for a binocular device according to claim 1, characterized in that, Determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the binocular device and the extrinsic parameters of the reference camera includes: Based on the rotation matrix between the cameras of the binocular device and the extrinsic parameters of the reference camera, the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of each camera of the binocular device are determined. Based on the rotation angle of the reference camera relative to the fixed line of sight, and the rotation angles of each camera in the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined.

6. The calibration method for a binocular device according to claim 5, characterized in that, The step of determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to a fixed line of sight and the rotation angles of each camera in the binocular device includes: The rotation matrix of the reference camera is determined based on the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of the reference camera in the binocular device. The rotation matrix of the non-reference camera is determined based on the rotation angle of the reference camera relative to the fixed line of sight and the rotation angle of each camera in the binocular device.

7. The calibration method for a binocular device according to claim 6, characterized in that, Determining the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to a fixed line of sight and the rotation angles of each camera in the binocular device includes: The rotation angle of the non-reference camera relative to the fixed line of view is determined based on the rotation angle of the reference camera relative to the fixed line of view and the rotation angle of each camera in the binocular device. The rotation matrix of the non-reference camera is determined based on the rotation angle of the non-reference camera relative to the fixed line of sight.

8. A calibration device for a binocular device, characterized in that, The device includes: The intrinsic and extrinsic parameter determination module is used to acquire multiple calibration board image pairs captured by the binocular device at different shooting angles, and based on the multiple calibration board image pairs, determine the rotation matrix, offset matrix and intrinsic parameters of each camera of the binocular device; The reference camera extrinsic parameter determination module is used to determine the reference camera in the binocular device, and, under the premise that the lens of the reference camera is parallel to the calibration board, to obtain multiple reference images of the calibration board using the reference camera, and to determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera. A transformation matrix determination module is used to determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the stereo device and the extrinsic parameters of the reference camera; and to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the stereo device; the rotation matrix of the reference camera is the rotation matrix of the reference camera rotating to a fixed viewing axis; the rotation matrix of the non-reference camera is the rotation matrix of the non-reference camera rotating to a fixed viewing axis. And a correction module, used to correct each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, to obtain a binocular device with a fixed line of sight.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the calibration method for the binocular device according to any one of claims 1 to 7.

Images