Binocular equipment correction method and device and computer equipment

Abstract

The invention relates to a binocular equipment correction method and device and computer equipment. The method comprises the following steps: acquiring a calibration plate image pair, and determining a rotation matrix and an offset matrix between cameras and internal parameters of the cameras based on the calibration plate image pair; on the premise that the lens of the reference camera is parallel to the calibration plate, obtaining a reference image of the calibration plate, and determining external parameters of the reference camera based on the reference image and internal parameters of the reference camera; determining a transformation matrix of the reference camera and a transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular equipment and the external parameters of the reference camera; and correcting the binocular equipment to obtain the binocular equipment with the fixed visual axis. By adopting the method, the problem that when a binocular endoscope corrected by an existing binocular equipment correction method is subjected to an edge uniformity test or a unit relative distortion test, a test card has trapezoidal distortion in an image, so that test data of edge uniformity, unit relative distortion and the like cannot meet test standards can be solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a calibration method, device and computer equipment for a binocular device. Background Art

[0002] With the development of science and technology, computer vision technology has developed rapidly. With the rapid development of computer vision technology, binocular devices are widely used. In the application process of binocular devices, in order to ensure its display effect, it needs to be calibrated first.

[0003] The existing calibration method of binocular equipment mainly obtains the external parameters of each camera based on the stereo calibration results of the binocular equipment, and then obtains the rotation matrix of the binocular camera based on the external parameters of each camera. When calibrating a binocular camera, one of the cameras is generally kept stationary, and the other camera rotates the angle of the rotation matrix, or the left and right cameras are each rotated along the positive and negative directions of the rotation vector. The angle of the rotation matrix makes the optical axes of the two cameras parallel. Using this calibration method for binocular equipment, there is a discrepancy between the directions of the optical axis and the visual axis of the image, which may affect the quality of the product. Taking a binocular endoscope as an example, the single-channel image of the binocular endoscope needs to be tested such as unit relative distortion and edge uniformity. If the visual axis and the optical axis are not parallel, when performing relevant parameter tests, after adjusting the distance and angle between the test card and the camera, the displayed test card image will have trapezoidal distortion, which is easy to cause the data of tests such as edge uniformity and unit relative distortion to fail to meet the test standards.

[0004] When calibrating a binocular endoscope using an existing binocular device calibration method, there is inconsistency between the directions of the optical axis and the visual axis of the image tested by the endoscope. When performing an edge uniformity test or a unit relative distortion test, after adjusting the position of the test card and the device according to the test standard, there is trapezoidal distortion in the image of the test card, resulting in the problem that the test data of edge uniformity, unit relative distortion, etc. do not meet the test standards. No effective solution has been proposed so far. Summary of the invention

[0005] Based on this, it is necessary to provide a correction method, device and computer equipment for a binocular device to address the above technical problems.

[0006] In a first aspect, the present application provides a method for calibrating a binocular device. The method comprises the following steps:

[0007] Acquire multiple calibration plate image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device based on the multiple calibration plate image pairs;

[0008] Determine a reference camera in the binocular device, and, under the premise that the lens of the reference camera is parallel to the calibration plate, use the reference camera to obtain multiple reference images of the calibration plate, and determine the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera;

[0009] Determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera;

[0010] Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis.

[0011] In one embodiment, the step of acquiring a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determining a rotation matrix, an offset matrix and an intrinsic parameter of each camera between the cameras of the binocular device based on the plurality of calibration plate image pairs, comprises the following steps:

[0012] Using each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances to obtain a plurality of calibration plate image pairs;

[0013] Based on the plurality of calibration plate image pairs, a preset binocular calibration algorithm is used to determine the rotation matrix, the offset matrix and the intrinsic parameters of each camera between the cameras of the binocular device.

[0014] In one embodiment, determining a reference camera in the binocular device and obtaining multiple reference images of the calibration plate using the reference camera under the premise that the lens of the reference camera is parallel to the calibration plate comprises the following steps:

[0015] Selecting any camera in the binocular device as the reference camera of the binocular device;

[0016] Adjusting the reference camera until the lens of the reference camera is parallel to the calibration plate, to obtain an adjusted reference camera;

[0017] The calibration plate is photographed using the adjusted reference camera to obtain a plurality of reference images of the calibration plate.

[0018] In one of the embodiments, determining the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera comprises the following steps:

[0019] Extracting features from each of the reference images respectively to obtain feature points in each of the reference images;

[0020] Based on the feature points in each of the reference images, feature point matching is performed between the reference images and the calibration plate to obtain matching results of feature points between the multiple reference images and the calibration plate;

[0021] Based on the matching results of the feature points between the multiple reference images and the calibration plate, and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined.

[0022] In one embodiment, determining the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera comprises the following steps:

[0023] Determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera;

[0024] Based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrices between the cameras of the binocular device, a transformation matrix of the reference camera and a transformation matrix of the non-reference camera are determined.

[0025] In one embodiment, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera comprises the following steps:

[0026] Based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera, determine the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of each camera of the binocular device;

[0027] Based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined.

[0028] In one embodiment, determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device comprises the following steps:

[0029] Determine the rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of the reference camera in the binocular device;

[0030] The rotation matrix of the non-reference camera is determined based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device.

[0031] In one embodiment, determining the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device comprises the following steps:

[0032] Determine the rotation angle of the non-reference camera relative to the fixed visual axis based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device;

[0033] The rotation matrix of the non-reference camera is determined based on the rotation angle of the non-reference camera relative to a fixed viewing axis.

[0034] In a second aspect, the present application also provides a calibration device for a binocular device. The device comprises:

[0035] An internal and external parameter determination module is used to obtain a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device based on the plurality of calibration plate image pairs;

[0036] A reference camera extrinsic parameter determination module is used to determine a reference camera in the binocular device, and on the premise that the lens of the reference camera is parallel to the calibration plate, use the reference camera to obtain multiple reference images of the calibration plate, and determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera;

[0037] A transformation matrix determination module, used to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera;

[0038] And a correction module is used to correct each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0039] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the binocular device calibration method described in the first aspect is implemented.

[0040] The above-mentioned binocular device calibration method, device and computer device determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device by using multiple calibration plate image pairs obtained by taking pictures of the binocular device at different angles, and then use the reference camera to take pictures of the calibration plate with a fixed visual axis to obtain multiple reference images, and use the reference images and the internal parameters of the reference camera to determine the external parameters of the reference camera in the binocular device, and then, according to the rotation matrix and offset matrix between the two cameras of the binocular device and the absolute external parameters of the reference camera, determine the various variable matrices of the two cameras in the binocular device corresponding to the fixed visual axis. The matrix is ​​converted, and then, each camera of the binocular device is calibrated according to the transformation matrix of each camera of the binocular device to obtain a binocular device with a fixed visual axis, so that the optical axis and the visual axis direction of the image of the binocular device are consistent, which solves the problem that when a binocular endoscope is calibrated based on an existing binocular device correction method, the directions of the optical axis and the visual axis of the image tested by the endoscope are inconsistent. When an edge uniformity test or a unit relative distortion test is performed, after the positions of the test card and the device are adjusted according to the test standard, the test card has trapezoidal distortion in the image, resulting in the problem that the test data of edge uniformity, unit relative distortion and other tests do not meet the test standards.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 A hardware structure block diagram of a terminal for a binocular device calibration method provided in an embodiment of the present application;

[0044] Figure 2 A flowchart of a method for calibrating a binocular device provided in one embodiment of the present application;

[0045] Figure 3 A flowchart of a binocular device calibration method provided in a preferred embodiment of the present application;

[0046] Figure 4 A structural block diagram of a calibration device for a binocular device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0048] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0049] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 : is a hardware structure block diagram of a terminal of the binocular device calibration method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0050] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the calibration method of the binocular device in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0052] In this embodiment, a calibration method for a binocular device is provided. Figure 2 is a flow chart of the calibration method of the binocular device of this embodiment. Figure 2 As shown, the process includes the following steps:

[0053] Step S210, obtaining a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determining the rotation matrix, offset matrix and intrinsic parameters of each camera of the binocular device based on the plurality of calibration plate image pairs.

[0054] Among them, the above-mentioned binocular device can be a binocular imaging device, for example, it can be a binocular endoscope or a binocular microscope and other devices. The above-mentioned binocular device includes two left and right cameras, each camera includes a lens, and the relative position between the two cameras is fixed. The above-mentioned calibration plate can be a flat plate with a fixed-pitch pattern array, wherein the above-mentioned fixed-pitch pattern can be a dot array, a checkerboard or other patterns with obvious corner point features. It should be noted that in order to ensure that there are enough feature points in the captured calibration plate image, it is necessary to ensure that the corner point features in the calibration plate reach the preset number of corner points. Specifically, the preset number of corner points can be set according to the specific situation. For example, the preset number of corner points can be 54.

[0055] The above-mentioned acquisition of multiple calibration plate image pairs obtained by the binocular device at different shooting angles can be to use each camera of the binocular device to simultaneously shoot the calibration plate at different shooting angles to obtain the calibration plate image pair. It should be noted that when using the binocular device to shoot the calibration plate, it is necessary to ensure that the binocular device shoots the calibration plate within the depth of field, and the clarity of the image in the obtained calibration plate image pair needs to meet the preset clarity requirements. By obtaining the calibration plate image pair, the calibration plate image pair can be used to calculate the external parameters (i.e., the rotation matrix and translation vector relative to a certain world coordinate system) and internal parameters of each camera of the binocular device, and the rotation matrix and offset matrix between each camera of the binocular device are determined through the external parameters and internal parameters of each camera of the binocular device.

[0056] Furthermore, the above rotation matrix can be the rotation relationship between the two camera coordinate systems of the binocular device. Specifically, it can be the rotation matrix of the two cameras rotated to the target coordinate system respectively. For example, the binocular device has a left camera and a right camera, and the rotation matrix between the left camera and the right camera includes a first rotation matrix from the left camera to the target coordinate system and a second rotation matrix from the right camera to the target coordinate system. That is, the rotation from the left camera coordinate system to the target coordinate system is the first rotation matrix. The rotation from the right camera coordinate system to the target coordinate system is the second rotation matrix. The above offset matrix, usually expressed as a translation vector, represents the translation relationship between the two camera coordinate systems.

[0057] The internal parameters of the camera may include the focal length, principal point position, distortion coefficient, etc. of the camera.

[0058] This step determines the positional relationship between the two cameras of the binocular device by determining the rotation matrix and the offset matrix between the cameras of the binocular device.

[0059] Step S220, determine the reference camera in the binocular device, and use the reference camera to obtain multiple reference images of the calibration plate under the premise that the lens of the reference camera is parallel to the calibration plate, and determine the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera.

[0060] In this step, the reference camera in the binocular device can be selected from any one of the binocular devices as the reference camera, and the other camera as the non-reference camera. It should be noted that when the lens of the reference camera is parallel to the calibration plate, the shooting direction of the reference camera is perpendicular to the plane of the calibration plate. In this case, the reference image of the calibration plate captured by the reference camera will not be perspectively deformed. Therefore, the calibration plate can be used to pre-set the fixed visual axis. When the calibration plate is fixed, at this time, adjusting the lens of the reference camera to be parallel to the calibration plate can ensure that the visual axis of the reference camera is a fixed visual axis (i.e., the center of the camera lens to the vertical direction of the calibration plate). When performing absolute external parameter calibration, if there is a rotation in a non-planar direction, the optical axis and the visual axis are not parallel. It should be noted that in the process of obtaining multiple reference images of the calibration plate using the reference camera, it is only necessary to keep the lens of the reference camera parallel to the calibration plate, and the distance between the reference camera and the calibration plate can be changed. As long as the calibration plate is within the depth of field of the reference camera, the reference image captured can meet the preset clarity requirements.

[0061] The above-mentioned determination of the external parameters of the reference camera based on multiple reference images can be performed by extracting features from each reference image to obtain feature points in each reference image, and then, according to the feature points in each reference image and the internal parameters of the reference camera, the Zhang Zhengyou calibration algorithm is applied to determine the external parameters of the reference camera. The external parameters of the reference camera are the absolute external parameters of the reference camera, specifically the rotation matrix and translation vector of the reference camera relative to the world coordinates of the calibration card.

[0062] Step S230, based on the rotation matrix and offset matrix between the cameras of the binocular device and the external parameters of the reference camera, determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0063] The transformation matrix of the reference camera may be a transformation relationship required when the reference camera is transformed to have a fixed viewing axis. The transformation matrix of the non-reference camera may be a transformation relationship required when the non-reference camera is transformed to have a fixed viewing axis.

[0064] In this step, the above-mentioned determination of the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the various cameras of the binocular device and the external parameters of the reference camera can be based on the rotation matrix, the offset matrix between the various cameras of the binocular device and the external parameters of the reference camera to determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and then, based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the various cameras of the binocular device, determine the transformation matrix of each camera of the binocular device.

[0065] Step S240: calibrate each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0066] In the above steps S210 to S240, the rotation matrix, offset matrix and internal parameters of each camera of the binocular device are determined by using a plurality of calibration plate image pairs obtained by taking pictures of the binocular device at different angles. Then, the calibration plate with a fixed visual axis is photographed by a reference camera to obtain a plurality of reference images. The reference images and the internal parameters of the reference camera are used to determine the external parameters of the reference camera in the binocular device. Then, the transformation matrices of the two cameras in the binocular device corresponding to the fixed visual axis are determined according to the rotation matrix and offset matrix between the two cameras of the binocular device and the absolute external parameters of the reference camera. Then, according to the transformation matrix of each camera of the binocular device, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis, so that the optical axis and the visual axis direction of the image of the binocular device are consistent, which solves the problem that the directions of the optical axis and the visual axis of the image tested by the endoscope are inconsistent when calibrating the binocular endoscope based on the existing binocular device correction method. When performing an edge uniformity test or a unit relative distortion test, after adjusting the position of the test card and the device according to the test standard, the test card has trapezoidal distortion in the image, resulting in the problem that the test data of edge uniformity, unit relative distortion and other tests do not meet the test standards.

[0067] In one embodiment, based on step S210, a plurality of calibration plate image pairs captured by the binocular device at different shooting angles are obtained, and based on the plurality of calibration plate image pairs, a rotation matrix, an offset matrix and an intrinsic parameter of each camera between the cameras of the binocular device are determined, including:

[0068] Step S212, using each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances to obtain multiple calibration plate image pairs.

[0069] The above-mentioned use of each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances to obtain multiple calibration plate image pairs can be achieved by adjusting the shooting angle and shooting position of the binocular device so that each camera of the binocular device synchronously shoots the calibration plate at different shooting angles and different shooting distances to obtain multiple calibration plate image pairs. The above-mentioned calibration plate image pairs are the matching results of the calibration plate images taken by different cameras at the same time.

[0070] Step S214, based on multiple calibration plate image pairs, using a preset binocular calibration algorithm, determine the rotation matrix, offset matrix and intrinsic parameters of each camera of the binocular device.

[0071] The preset dual-target calibration algorithm mentioned above may be one or more algorithms such as Zhang's calibration method, ICP (Iterative Closest Point) algorithm, DLT (Direct Linear Transform) algorithm, etc.

[0072] Specifically, the above-mentioned method is based on multiple calibration plate image pairs and uses a preset binocular positioning algorithm to determine the rotation matrix, offset matrix and intrinsic parameters of each camera between the cameras of the binocular device. It can be that according to the multiple calibration plate image pairs, feature points of each image in the calibration plate image pair are extracted, and then based on the extracted feature points, feature points of each calibration plate image pair are matched to obtain feature point matching results of each calibration plate image pair. According to the feature point matching results of each calibration plate image pair, the preset binocular positioning algorithm is used to determine the rotation matrix, offset matrix and intrinsic parameters of each camera between the cameras of the binocular device.

[0073] In the above steps S212 to S214, by using each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances, a plurality of calibration plate image pairs are obtained, and then, according to the calibration plate image pairs, the rotation matrix, the offset matrix and the intrinsic parameters of each camera between the cameras of the binocular device are determined. By determining the rotation matrix, the offset matrix and the intrinsic parameters of each camera between the cameras of the binocular device, it is convenient to subsequently determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0074] Specifically, in one embodiment, the above step S220 determines a reference camera in the binocular device, and obtains multiple reference images of the calibration plate using the reference camera under the premise that the lens of the reference camera is parallel to the calibration plate, including:

[0075] Step S221, selecting any camera in the binocular device as a reference camera of the binocular device.

[0076] Step S222, adjusting the reference camera until the lens of the reference camera is parallel to the calibration plate, thereby obtaining an adjusted reference camera.

[0077] Step S223, photographing the calibration plate using the adjusted reference camera to obtain multiple reference images of the calibration plate.

[0078] It should be noted that when using the reference camera to photograph the calibration plate, it is only necessary to ensure that the lens of the reference camera is parallel to the calibration plate, and the distance between the reference camera and the calibration plate can be changed.

[0079] In the above steps S221 to S223, by selecting a reference camera in the binocular device and adjusting the reference camera until the lens of the reference camera is parallel to the calibration plate, it is convenient to obtain a reference image of the calibration plate under the premise that the lens of the reference camera is parallel to the calibration plate, so as to facilitate the subsequent use of the reference image of the calibration plate to obtain the external parameters of the reference camera.

[0080] In addition, in one embodiment, in the above step S220, determining the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera includes:

[0081] Step S224, extracting features from each reference image to obtain feature points in each reference image.

[0082] The above method of determining the feature points of the reference image may be to use a feature point detection algorithm such as a SIFT (Scale Invariant Feature Transform) algorithm or a Harris algorithm, or other feature point detection algorithms, as long as the feature points of each reference image can be determined. This embodiment does not specifically limit the method for obtaining the feature points of the reference image.

[0083] Step S225 , based on the feature points in each reference image, feature point matching is performed between the reference image and the calibration plate to obtain matching results of the feature points between each reference image and the calibration plate.

[0084] Step S226, determining the external parameters of the reference camera based on the matching results of the feature points between each reference image and the calibration plate, and the internal parameters of the reference camera.

[0085] Because the collective shape of the calibration plate is fixed, each feature point in the calibration plate has a known world coordinate. Based on the matching results between the feature points of each reference image and the calibration plate, and the position of each feature point in the calibration plate in the reference image, a correspondence can be established between each feature point in the calibration plate and the feature point in the reference image. Furthermore, the calibration plate coordinate system can be used as the world coordinate system, and the PNP (Perspective-n-Point) algorithm can be used to solve the extrinsic parameters between the reference camera and the calibration plate to obtain the extrinsic parameters of the reference camera.

[0086] According to Zhang Zhengyou's monocular camera calibration method, after calculating the homography matrix, the external parameters (rotation + translation matrix) of each image are calculated based on the internal parameter data obtained in the binocular calibration. The external parameters of the above image can be the absolute external parameters of the image. Because the requirement for capturing the image is that the lens of the reference camera is parallel to the calibration plate, the rotation matrix parameters in the absolute external parameters obtained from different reference images are relatively consistent. After obtaining each absolute external parameter, the optimization method is used to obtain the rotation matrix in the optimal absolute external parameter.

[0087] In the above steps S224 to S226, feature points in each reference image are obtained by respectively extracting features from each reference image, and then the external parameters of the reference camera are determined by using the feature points of each reference image and the internal parameters of the reference camera. The determination of the external parameters of the reference camera facilitates the subsequent determination of the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the external parameters of the reference camera.

[0088] In one embodiment, based on step S230, based on the rotation matrix and offset matrix between the cameras of the binocular device and the external parameters of the reference camera, the transformation matrix of the reference camera and the transformation matrix of the non-reference camera are determined, including:

[0089] Step S232, based on the rotation matrices between the cameras of the binocular device and the external parameters of the reference camera, determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0090] The above-mentioned determination of the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the various cameras of the binocular device and the external parameters of the reference camera can be based on the rotation matrix between the various cameras of the binocular device and the external parameters of the reference camera, to determine the rotation angle of the reference camera relative to the fixed visual axis and the various rotation angles of the various cameras of the binocular device, and then determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0091] The rotation matrix of the reference camera may be a rotation matrix of the reference camera rotated to a fixed visual axis, and the rotation matrix of the non-reference camera may be a rotation matrix of the non-reference camera rotated to a fixed visual axis.

[0092] Step S234, based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the binocular device, determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera.

[0093] The offset matrix is ​​known. According to the correction principle of row alignment in epipolar correction, the row alignment matrix is ​​obtained through the offset matrix, and the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are acted on to obtain the transformation matrix of the reference camera and the non-reference camera.

[0094] In the above steps S232 to S234, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined through the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera. Then, the transformation matrix of each camera of the binocular device is determined according to the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, as well as the offset matrix between the cameras of the binocular device, so as to facilitate the subsequent correction of the binocular device according to the transformation matrix of each camera of the binocular device, so that the binocular device has a fixed visual axis.

[0095] In addition, in one embodiment, the above step S232, based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera, determines the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, including:

[0096] Step S2322, based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera, determine the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of each camera of the binocular device.

[0097] Among them, the rotation angle of the above-mentioned reference camera relative to the fixed visual axis can be the rotation angle of the reference camera rotated to the fixed visual axis in various directions, and the above-mentioned various directions can be the x-axis direction, y-axis direction and z-axis direction of the world coordinate system. The rotation angle of each camera of the above-mentioned binocular device can be the rotation angle of each camera coordinate system of the binocular device rotated to the target coordinate system. For example, the binocular device has a left camera and a right camera, and the rotation angles of the left camera and the right camera include a first rotation angle from the left camera to the target coordinate system and a second rotation angle from the right camera to the target coordinate system. Among them, the rotation angle from the left camera coordinate system to the target coordinate system in various directions is the first rotation angle. The rotation angle from the right camera coordinate system to the target coordinate system in various directions is the second rotation angle. It should be noted that the above-mentioned target coordinate system can be a coordinate system corresponding to the minimum angle when the left and right roads are aligned relative to the rotating coordinate system or a fixed coordinate system. For example, a coordinate system in which the optical axis and the visual axis are parallel.

[0098] The calculation process for determining the rotation angle of the reference camera relative to the fixed viewing axis based on the external parameters of the reference camera is as follows:

[0099]

[0100] Among them, R is the rotation matrix in the external parameters of the reference camera, r ab Represents the elements of the rotation matrix in the extrinsic parameters of the reference camera, where a represents the ath row of the rotation matrix in the extrinsic parameters of the reference camera, and b represents the bth column of the rotation matrix in the extrinsic parameters of the reference camera. For example, r 11 Represents the first row and first column of the rotation matrix in the external parameters of the reference camera.

[0101] The Euler angle deviations of the rotation matrix in the external parameters of the reference camera in the x-axis, y-axis and z-axis directions can be expressed as:

[0102]

[0103] in, is the rotation angle of the reference camera around the x-axis relative to the fixed visual axis, θ is the rotation angle of the reference camera around the y-axis relative to the fixed visual axis, is the rotation angle of the reference camera around the z-axis relative to the fixed viewing axis.

[0104] The expression of the rotation matrix in the external parameters of the reference camera expressed by the rotation angles in three directions is:

[0105]

[0106] By solving the Euler angle rotation matrix formula, we can get the rotation angle of the reference camera around different directions relative to the fixed visual axis. The solution results are as follows:

[0107]

[0108] When the rotation matrices between the cameras of the binocular device are determined, the first rotation matrix and the second rotation matrix can be determined based on the rotation matrices between the cameras of the binocular device. When the first rotation matrix and the second rotation matrix are determined, the same method as determining the rotation angle of the reference camera relative to the fixed visual axis in different directions can be adopted. According to the first rotation matrix, the first rotation angle of the left camera in different directions relative to the target coordinate system in the binocular device can be determined. According to the second rotation matrix, the rotation angle of the right camera in different directions relative to the target coordinate system in the binocular device can be determined.

[0109] Step S2324, based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device, determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera.

[0110] In the above steps S2322 to S2324, by determining the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined according to the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device. This facilitates the subsequent correction of each camera of the binocular device according to the rotation matrix of the reference camera and the rotation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0111] Further, in one embodiment, the above step S2324, based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device, determines the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, including:

[0112] Step S1, determining a rotation matrix of a reference camera based on a rotation angle of the reference camera relative to a fixed visual axis and a rotation angle of the reference camera in a binocular device.

[0113] When the rotation angle of the reference camera around different directions relative to the fixed visual axis is determined, the reference camera can be adjusted to rotate around the x-axis and the y-axis according to the fixed visual axis, and rotate around the z-axis according to the original direction (under the premise that the lens of the reference camera is parallel to the calibration plate, the rotation around the z-axis only affects the rotation in the plane direction. In order to reduce cropping, the rotation in the plane direction is not considered when correcting the rotation matrix), and the rotation matrix of the reference camera is obtained. The specific calculation process is as follows:

[0114]

[0115] Where R′ is the rotation matrix of the reference camera. is the rotation angle around the z-axis in the first rotation angle.

[0116] In this embodiment, the left camera is used as a reference image and the right camera is used as a non-reference image for calculation. The right image can also be used as a reference image and the left image as a non-reference image. This embodiment is not specifically limited here, and the calculation method is the same as the method of using the left image as the reference image.

[0117] Step S2, determining a rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of each camera in the binocular device.

[0118] The process of calculating the rotation matrix of a non-reference camera is as follows:

[0119]

[0120] Among them, r r ′ is the rotation matrix of the non-reference camera, is the rotation angle of the non-reference camera around the x-axis relative to the fixed viewing axis, θ x is the rotation angle of the non-reference camera around the y-axis relative to the fixed viewing axis, is the rotation angle around the z-axis in the second rotation angle.

[0121] In this embodiment, the left camera is used as a reference image and the right camera is used as a non-reference image for calculation. The right image can also be used as a reference image and the left image as a non-reference image. This embodiment is not specifically limited here, and the calculation method is the same as the method of using the left image as the reference image.

[0122] In the above steps S1 to S2, the rotation matrix of the reference camera is determined based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of the reference camera in the binocular device, and the rotation matrix of the non-reference camera is determined based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device. By determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, it is convenient to calibrate each camera of the binocular device according to the rotation matrix of the reference camera and the rotation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0123] Specifically, in one embodiment, based on step S2, based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of the cameras in the binocular device, the rotation matrix of the non-reference camera is determined, including:

[0124] Step S22, based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device, determine the rotation angle of the non-reference camera relative to the fixed visual axis.

[0125] The calculation process of the rotation angle of the above non-reference camera relative to the fixed viewing axis is as follows:

[0126]

[0127] in, is the rotation angle around the x-axis in the second rotation angle, is the rotation angle around the x-axis in the first rotation angle.

[0128] θ x =θ r +(θ-θ l )

[0129] Among them, θ r is the rotation angle around the y-axis in the second rotation angle, θ l is the rotation angle around the y-axis in the first rotation angle.

[0130]

[0131] in, is the rotation angle of the non-reference camera around the z-axis relative to the fixed viewing axis.

[0132] Step S24: determining a rotation matrix of the non-reference camera based on the rotation angle of the non-reference camera relative to the fixed viewing axis.

[0133] In the above steps S22 to S24, the rotation angle of the non-reference camera relative to the fixed visual axis is determined based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device. Then, the rotation matrix of the non-reference camera is determined according to the rotation angle of the non-reference camera relative to the fixed visual axis and the rotation angle of the non-reference camera in the binocular device. The confirmation of the rotation matrix of the non-reference camera facilitates the subsequent correction of each camera of the binocular device to obtain a binocular device with a fixed visual axis.

[0134] The present embodiment is described and illustrated below through preferred embodiments.

[0135] Figure 3 Flowchart of a binocular device calibration method provided by a preferred embodiment of the present application. Figure 3 As shown, the calibration method of the binocular device includes the following steps:

[0136] Step S301, using each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances to obtain multiple calibration plate image pairs;

[0137] Step S302, based on multiple calibration plate image pairs, using a preset binocular calibration algorithm, determining the rotation matrix, offset matrix and internal parameters of each camera of the binocular device;

[0138] Step S303, determining a reference camera in the binocular device, and using the reference camera to obtain multiple reference images of the calibration plate under the premise that the lens of the reference camera is parallel to the calibration plate, and determining the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera;

[0139] Step S304, determining the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of each camera of the binocular device based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera;

[0140] Step S305, determining a rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed visual axis;

[0141] Step S306, determining a rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device;

[0142] Step S307, determining the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrix between the cameras of the binocular device;

[0143] Step S308: calibrate each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0144] In the above steps S301 to S308, the rotation matrix, offset matrix and internal parameters of each camera of the binocular device are determined by using a plurality of calibration plate image pairs obtained by taking pictures at different angles of the binocular device. Then, the calibration plate with a fixed visual axis is photographed by a reference camera to obtain a plurality of reference images. The reference images and the internal parameters of the reference camera are used to determine the external parameters of the reference camera in the binocular device. Then, the transformation matrices of the two cameras in the binocular device corresponding to the fixed visual axis are determined according to the rotation matrix and offset matrix between the two cameras of the binocular device and the external parameters of the reference camera. Array, then, according to the transformation matrix of each camera of the binocular device, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis, so that the optical axis and the visual axis direction of the image of the binocular device are consistent, which solves the problem that when a binocular endoscope is calibrated based on an existing binocular device correction method, the directions of the optical axis and the visual axis of the image tested by the endoscope are inconsistent. When an edge uniformity test or a unit relative distortion test is performed, after the positions of the test card and the device are adjusted according to the test standard, the test card has trapezoidal distortion in the image, resulting in the problem that the test data of edge uniformity, unit relative distortion and other tests do not meet the test standards.

[0145] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0146] Based on the same inventive concept, a correction device for a binocular device is also provided in this embodiment, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0147] In one embodiment, Figure 4 is a structural block diagram of a calibration device for a binocular device provided in an embodiment of the present application, such as Figure 4 As shown, the correction device of the binocular device includes:

[0148] The internal and external parameter determination module 42 is used to obtain a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device based on the plurality of calibration plate image pairs;

[0149] The reference camera external parameter determination module 44 is used to determine the reference camera in the binocular device, and obtain multiple reference images of the calibration plate using the reference camera under the premise that the lens of the reference camera is parallel to the calibration plate, and determine the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera;

[0150] A transformation matrix determination module 46 is used to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera;

[0151] And a correction module 48 is used to correct each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

[0152] The calibration device of the binocular device uses a plurality of calibration plate image pairs obtained by taking pictures of the binocular device at different angles to determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device, and then uses a reference camera to take pictures of the calibration plate with a fixed visual axis to obtain a plurality of reference images, and uses the reference images and the internal parameters of the reference camera to determine the external parameters of the reference camera in the binocular device, and then, according to the rotation matrix and offset matrix between the two cameras of the binocular device and the absolute external parameters of the reference camera, determines the transformation matrices of the two cameras in the binocular device corresponding to the fixed visual axis, and then Finally, according to the transformation matrix of each camera of the binocular device, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis, so that the optical axis and the visual axis of the image of the binocular device are consistent, which solves the problem that when calibrating the binocular endoscope based on the existing binocular device correction method, the directions of the optical axis and the visual axis of the image tested by the endoscope are inconsistent. When performing an edge uniformity test or a unit relative distortion test, after adjusting the position of the test card and the device according to the test standard, the test card has trapezoidal distortion in the image, resulting in the problem that the test data of edge uniformity, unit relative distortion and other tests do not meet the test standards.

[0153] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0154] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any one of the binocular device calibration methods in the above embodiments is implemented.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A calibration method for a binocular device, characterized in that: The method comprises: Acquire multiple calibration plate image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device based on the multiple calibration plate image pairs; Determine a reference camera in the binocular device, and, under the premise that the lens of the reference camera is parallel to the calibration plate, use the reference camera to obtain multiple reference images of the calibration plate, and determine the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera; Determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera; Based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera, each camera of the binocular device is calibrated to obtain a binocular device with a fixed visual axis.

2. The calibration method of the binocular device according to claim 1, characterized in that: The step of acquiring a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determining a rotation matrix, an offset matrix and an internal parameter of each camera between the binocular device based on the plurality of calibration plate image pairs, includes: Using each camera of the binocular device to shoot the calibration plate at different shooting angles and different shooting distances to obtain a plurality of calibration plate image pairs; Based on the plurality of calibration plate image pairs, a preset binocular calibration algorithm is used to determine the rotation matrix, the offset matrix and the intrinsic parameters of each camera between the cameras of the binocular device.

3. The calibration method of binocular device according to claim 1, characterized in that: The step of determining a reference camera in the binocular device and obtaining a plurality of reference images of the calibration plate by using the reference camera under the premise that the lens of the reference camera is parallel to the calibration plate comprises: Selecting any camera in the binocular device as the reference camera of the binocular device; Adjusting the reference camera until the lens of the reference camera is parallel to the calibration plate, to obtain an adjusted reference camera; The calibration plate is photographed using the adjusted reference camera to obtain a plurality of reference images of the calibration plate.

4. The calibration method of a binocular device according to claim 1, characterized in that: The determining the external parameters of the reference camera based on the multiple reference images and the internal parameters of the reference camera includes: Extracting features from each of the reference images respectively to obtain feature points in each of the reference images; Based on the feature points in each of the reference images, feature point matching is performed between the reference image and the calibration plate to obtain a matching result of the feature points between each of the reference images and the calibration plate; Based on the matching results of the feature points between each of the reference images and the calibration plate, and the intrinsic parameters of the reference camera, the extrinsic parameters of the reference camera are determined.

5. The calibration method of binocular device according to claim 1, characterized in that: The step of determining the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera includes: Determine the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera; Based on the rotation matrix of the reference camera and the rotation matrix of the non-reference camera, and the offset matrices between the cameras of the binocular device, a transformation matrix of the reference camera and a transformation matrix of the non-reference camera are determined.

6. The calibration method of binocular device according to claim 5, characterized in that: The step of determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera includes: Based on the rotation matrix between the cameras of the binocular device and the external parameters of the reference camera, determine the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of each camera of the binocular device; Based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device, the rotation matrix of the reference camera and the rotation matrix of the non-reference camera are determined.

7. The calibration method of binocular device according to claim 6, characterized in that: The determining the rotation matrix of the reference camera and the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera of the binocular device comprises: Determine the rotation matrix of the reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angle of the reference camera in the binocular device; The rotation matrix of the non-reference camera is determined based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device.

8. The binocular device calibration method according to claim 7, characterized in that: The determining the rotation matrix of the non-reference camera based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device comprises: Determine the rotation angle of the non-reference camera relative to the fixed visual axis based on the rotation angle of the reference camera relative to the fixed visual axis and the rotation angles of each camera in the binocular device; The rotation matrix of the non-reference camera is determined based on the rotation angle of the non-reference camera relative to a fixed viewing axis.

9. A calibration device for a binocular device, characterized in that: The device comprises: An internal and external parameter determination module is used to obtain a plurality of calibration plate image pairs captured by the binocular device at different shooting angles, and determine the rotation matrix, offset matrix and internal parameters of each camera of the binocular device based on the plurality of calibration plate image pairs; A reference camera extrinsic parameter determination module is used to determine a reference camera in the binocular device, and on the premise that the lens of the reference camera is parallel to the calibration plate, use the reference camera to obtain multiple reference images of the calibration plate, and determine the extrinsic parameters of the reference camera based on the multiple reference images and the intrinsic parameters of the reference camera; A transformation matrix determination module, used to determine the transformation matrix of the reference camera and the transformation matrix of the non-reference camera based on the rotation matrix and the offset matrix between the cameras of the binocular device and the external parameters of the reference camera; And a correction module is used to correct each camera of the binocular device based on the transformation matrix of the reference camera and the transformation matrix of the non-reference camera to obtain a binocular device with a fixed visual axis.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the binocular device calibration method according to any one of claims 1 to 8 are implemented.

Images